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TRAINING  TO  IMPROVE  THE  ORGANIZATION  OF  TACTICAL  KNOWLEDGE: 
AN  EVALUATION  OF  THE  ARMOR  TACTICAL  CONCEPTS  TUTOR 


Introduction 

The  Armor  Officer  Basic  Course  (AOBC)  is  a  16-week  (plus  two  days)  course 
of  instruction  for  training  newly  commissioned  armor  officers.  AOBC  trains 
officers  in  two  different  and  complex  duties:  that  of  tank  commander  and  that 
of  platoon  leader.  In  addition  to  providing  skills  and  knowledge  required  to 
perform  these  two  critical  duties,  AOBC  is  important  to  armor's  mission  in 
that  it  provides  the  foundation  for  officer  training  in  the  Armor  Branch.  In 
fact,  it  is  the  only  formal  course  work  that  the  second  lieutenant  will 
receive  prior  to  being  assigned  as  a  platoon  leader.  It  is  crucial  that  AOBC 
instruction  be  effective  in  developing  the  basic  knowledge  and  skills  that  a 
second  lieutenant  needs  to  command  a  tank  and  to  lead  an  armor  platoon. 

Transfer  of  Training  Problems  in  AOBC 

Kern  (cited  in  Bryant  et  al . ,  1987)  studied  AOBC  as  it  was  conducted  in 
1984  and  conceptually  divided  the  course  into  two  phases.  In  the  first  phase 
(the  first  two-thirds  of  the  course),  students  learned  basic  facts  and 
procedures  that  focused  on  characteristics,  capabilities,  maintenance,  and 
operation  of  equipment.  Because  most  new  lieutenants  had  little  or  no  armor 
tactical  background,  this  focus  fostered  the  perception  that  training  had  only 
some  obscure  relevance  to  whatever  platoon  leaders  might  need  to  know  and  do 
in  planning  and  conducting  tactical  operations.  Instruction  in  the  second 
phase  (the  last  third  of  the  course)  focused  on  teaching  the  student  tactical 
concepts  employed  in  maneuvering  the  tank  platoon  to  accomplish  various 
offensive  operations  and  in  preparing  and  defending  battle  positions. 

Learning  these  tactical  concepts  and  how  to  employ  them  during  practice  and 
during  performance  tests  required  the  student  to  recall  and  appropriately 
apply  facts  and  procedures  taught  in  the  first  phase.  Examples  of  facts  and 
procedures  that  are  relevant  to  tactical  concepts  include  the  effective  range 
of  weapons,  the  CEOI  for  encoding  and  decoding  radio  messages,  hand-and-arm 
signals,  tank  formations,  and  techniques  of  movement.1 

Student  test  scores  from  phases  1  and  2  documented  the  steep  decrease  in 
performance  when  the  focus  of  instruction  shifted  from  learning  relatively 
discrete  facts  and  procedures  to  learning  how  to  integrate  this  diverse 
knowledge  with  the  new  knowledge  being  learned  in  phase  2.  Whereas  first- 
attempt  pass  rates  for  the  first  phase  of  AOBC  were  greater  than  93  percent, 
the  corresponding  performance  for  the  second  phase  of  training  dropped  to 
approximately  70  percent.  The  types  of  errors  made  suggested  that  students 
were  finding  it  especially  difficult  to  recall  and  appropriately  integrate 


‘Since  the  1984  observations,  AOBC  has  been  extended  from  15  weeks  to 
16  weeks,  2  days.  In  addition,  hand-and-arm  signals  and  techniques  of 
movement  training  were  rescheduled  to  occur  in  the  initial  part  of  the  second, 
or  tactical,  training  phase. 
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knowledge  learned  in  the  first  phase  of  the  instruction  into  the  tactical 
context  of  the  second  phase. 

An  extensive  body  of  research  literature  suggests  that  students  would  be 
expected  to  exhibit  failures  in  retrieving  and  applying  knowledge  learned  in 
one  context  to  solving  problems  encountered  in  a  different  context  (e.g., 
Bransford,  Sherwood,  Vye,  &  Rieser,  1986;  Barsalou,  1988;  Franks,  Bransford,  & 
Auble,  1982).  When  the  AOBC  was  designed  in  1984,  very  few  explicit  linkages 
were  made  in  phase  1  between  the  facts  and  procedures  being  trained  and  their 
significance  to  the  planning  or  conducting  of  tactical  operations.  Since  that 
time  the  Armor  School  has  worked  on  improving  the  linkages  between  knowledge 
taught  in  phase  1  and  the  tactical  training  taught  in  phase  2.  However,  the 
extensive  amount  and  diversity  of  information  presented  in  phase  1  are  too 
large  to  be  easily  retained  and  retrieved,  especially  if  the  student  only 
receives  the  information  in  the  classroom  and  does  not  use  it  in  any 
operational  context  until  he  reaches  phase  2.  In  other  words,  the  transfer 
problem  may  be  due  to  lack  of  sufficient  exercises  in  establishing  the 
relationships  between  facts  and  procedures  taught  in  phase  1  to  tactical 
objectives  taught  in  phase  2.  In  recognition  of  these  problems,  the  tactical 
instructors  provide  additional  training  in  phase  2  consisting  of  one-on-one 
practice  in  conducting  tactical  operations  on  terrain  boards  as  well  as 
additional  tutoring  during  the  final  field  exercises.  While  this  approach  is 
recognized  as  effective,  it  is  clearly  labor  intensive  and  limited  by  the 
number  of  available  instructors.  Furthermore,  even  greater  effectiveness  might 
be  achieved  if  training  were  designed  so  that  the  student  recognized  the 
need  for  retrieving  appropriate  phase  1  knowledge  when  he  enters  phase  2 
tactical  training. 

A  second  contributor  to  the  transfer  problem  relates  to  the  students'  use 
of  ineffective  cognitive  strategies  for  learning  and  storing  information 
presented  in  long-term  memory.  Despite  the  considerable  information 
processing  demands  of  the  course,  Kern  (cited  in  Stoddard,  Kern,  &  Emerson, 
1986)  observed  that  many  AOBC  students  persisted  in  using  simple,  inefficient 
strategies  (primarily,  rote  memorizing)  to  acquire  and  retain  information.  As 
a  result,  these  students  did  not  interrelate  facts  and  procedures  taught  in 
phase  1  with  possible  implications  for  their  significance  to  later  performance 
requirements.  Thus,  when  they  entered  the  tactical  context  of  phase  2,  they 
were  unable  to  retrieve  information  that  was  stored  by  rote  memorization  and 
had  to  relearn  it  in  the  new  context.  Stoddard  et  al.  (1986)  hypothesized 
that,  in  contrast  to  students,  experts  in  tactics  ( i . e . ,  senior  NCOs  and 
officers)  are  able  to  readily  apply  tactical  information  to  actual  problems 
because  their  knowledge  of  tactics-related  facts  and  procedures  is  organized 
in  a  manner  that  facilitates  effective  retrieval.  This  speculation  was  based 
on  a  growing  body  of  research  showing  differences  between  experts  and  novices 
in  knowledge  organization  and  processing  in  a  variety  of  content  areas  (e.g., 
Glaser,  1984).  The  implication  from  this  hypothesis  is  that  AOBC  training 
should  be  designed  to  encourage  and  assist  the  student  in  developing  a  more 
highly  interrelated  knowledge  base  starting  early  in  AOBC. 
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Development  of  the  Armor  Tactical  Concepts  Tutor 

The  Armor  Tactical  Concepts  Tutor  (ARTACT)  was  developed  by  ARI  with  the 
assistance  of  the  Cognitive  Engineering  Design  and  Research  Team  from  the  Los 
Alamos  National  Laboratory.  The  objective  in  designing  ARTACT  was  to  provide 
the  Armor  School  with  prototype  computer-tutor  programs  that  would  cause  the 
student  to  perform  the  types  of  cognitive  processing  needed  to  develop  a  more 
highly  interrelated  knowledge  base  starting  early  in  AOBC.  ARTACT  is  a 
prototype  system  in  that  it  is  intended  only  to  provide  a  sufficient  number  of 
programs  needed  to  demonstrate  its  potential  and,  if  accepted,  to  provide  a 
shell  that  could  accept  additional  programs. 

The  ARTACT  programs  are  designed  to  be  inserted  at  various  points  during 
the  facts  and  procedures  training  (phase  1)  that  precedes  the  armor  tactical 
training  (phase  2).  The  objective  of  these  programs  is  to  train  the  student 
to  interrelate  appropriate  facts  and  procedures  taught  in  phase  1  within  the 
context  of  tactical  operations  before  he  reaches  phase  2  training.  To 
accomplish  this,  four  interactive  videodisc  programs  were  developed. 

Identified  by  their  tactical  context,  these  four  are  described  as  follows: 

1.  Planning  Field  Artillery  Fires:  Planning  target  locations  and  type 
of  fires  to  defend  a  battle  position. 

2.  Techniques  of  Movement,  Formations  and  Battle  Drills:  Scenario-based 
planning  and  execution  of  movement  from  battle  position  to  objective. 

3.  Offensive  Operations:  Scenario-based  planning  and  execution  with 
responses  to  unexpected  problems. 

4.  Defensive  Operations:  Scenario-based  planning  and  execution  with 
responses  to  unexpected  problems. 

These  programs  put  the  students  in  a  simulated  context  of  planning  and 
conducting  tactical  operations.  The  scenarios  used  are  designed  to  require 
students  to  recognize  the  need  for  retrieving  and  applying  appropriate  facts 
and  procedures  taught  in  preceding  classes  of  phase  1.  Thus,  the  primary 
objective  of  ARTACT  is  to  augment  the  regular  phase  1  instruction  by  training 
students  to  interrelate  knowledge  learned  in  phase  1  with  knowledge  to  be 
taught  in  phase  2.  Since  the  students  will  receive  these  programs  before 
receiving  tactical  training,  it  was  necessary  that  they  provide  tutoring  on 
basic  tactical  knowledge  when  students  are  otherwise  unable  to  proceed.  As  a 
result,  a  secondary  objective  of  ARTACT  is  to  tutor  students  on  basic  tactical 
knowledge. 

ARTACT  programs  were  given  a  full  doctrinal  review  by  subject  matter 
experts  from  the  Tank  Platoon  Tactics  committee  in  September  1987,  and  changes 
required  were  made  prior  to  initiation  of  this  evaluation. 
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Purposes  of  the  Evaluation 


The  overall  purpose  of  the  present  study  was  to  conduct  a  small-scale 
evaluation  of  ARTACT.  The  evaluation  focused  principally  on  the  effects  of 
ARTACT  on  student  performance.  To  determine  these  effects,  the  performance  of 
a  group  of  students  who  received  ARTACT  instruction  was  compared  to  the 
performance  of  another  group  who  did  not  receive  ARTACT  instruction  using  a 
pretest/posttest  design.  This  evaluation  represented  the  first  u.j  of  ARTACT 
in  an  instructional  setting.  Therefore,  an  additional  purpose  of  the 
evaluation  was  to  identify  problems  in  operating  ARTACT  in  a  realistic  setting 
and  to  assess  subjective  reactions  to  the  system.  The  data  obtained  from  the 
evaluation  may  be  grouped  according  to  the  following  four  sets  of  evaluation 
issues. 

Effects  on  Tactical  Knowledges  and  Skills 

This  issue  related  to  the  question  of  whether  or  not  ARTACT  training, 
conducted  prior  to  the  regular  tactical  training,  resulted  in  greater 
knowledge  for  planning  and  conducting  tactical  operations.  Special  tests  on 
the  subset  of  AOBC  tactical  skills  and  knowledges  addressed  by  ARTACT  were 
administered  to  all  participants  near  the  beginning  and  at  the  end  of  AOBC. 

Effects  on  Organization  of  Tactical  Knowledges 

This  issue  reflects  an  important  assumption  underlying  the  rationale 
used  in  the  design  of  ARTACT.  However,  there  are  no  well  established  methods 
for  assessing  organization  of  a  person's  knowledge  of  a  given  topical  domain 
that  can  serve  as  evaluative  criteria.  Consequently,  new  techniques  were 
developed  to  assess  the  organization  of  students'  knowledge  structures.  If 
feasible,  these  techniques  could  provide  empirical  evidence  on  the 
relationship  between  the  quality  of  students'  performance  and  the  types  of 
organization  of  knowledge  they  had  achieved.  Because  of  their  experimental 
status,  any  findings  concerning  the  effectiveness  of  ARTACT  that  are  based  on 
these  techniques  should  be  regarded  as  suggestive. 

Effects  on  Performance  in  AOBC 


This  issue  addressed  the  effects  of  ARTACT  training  on  student 
performance  in  AOBC.  Two  types  of  AOBC  measures  were  obtained:  (a)  overall 
performance  in  AOBC  based  on  the  student's  class  standing  (percentile  rank) 
and  average  academic  grades,  and  (b)  scores  from  the  Armor  Performance  Test, 
an  examination  administered  in  the  14th  week  of  AOBC  that  focuses  on  tactical 
skills  and  knowledges. 

Formative  Issues 


Because  this  study  is  the  first  field  test  of  ARTACT,  two  formative 
evaluation  issues  were  addressed:  (a)  the  identification  of  ARTACT  software 
and  hardware  problems  evident  from  operating  the  system  in  AOBC,  and  (b)  the 
assessment  of  subjective  reactions  of  AOBC  students  to  the  ARTACT  system. 
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Method 


Experimental  Participants 

Participants  in  the  present  experiment  were  students  from  six 
consecutive  AOBC  classes  undergoing  training  in  FY  1988.  Names  of  12  or  more 
volunteers  from  each  of  the  six  classes  were  provided  to  the  experimenters  who 
randomly  assigned  six  students  to  the  experimental  group  and  six  to  the 
control  group.  Students  in  the  experimental  condition  received  training  on 
ARTACT  whereas  students  in  the  control  condition  did  not.  Hardware  and 
software  problems  precluded  implementing  all  four  ARTACT  modules  in  the  first 
class;  consequently,  the  data  from  the  first  class  were  not  used.  Of  the 
remaining  five  classes  (60  students),  two  students  assigned  to  the  exper¬ 
imental  group  withdrew  from  ARTACT  for  medical  or  personal  reasons.  Six  other 
students  (3  experimentals,  3  controls)  failed  to  show  up  for  the  posttest  and 
graduated  from  AOBC  before  they  could  be  tested.  Therefore,  the  pre-post 
comparisons  were  based  on  a  total  of  52  AOBC  students,  25  in  the  experimental 
group  and  27  in  the  control.  Variations  to  these  sample  sizes  are  noted  in 
relevant  sections  of  the  Results. 

All  students  were  initially  given  a  questionnaire  to  determine  their 
background  and  experience.  A  copy  of  this  questionnaire  is  provided  at 
Appendix  A-l,  and  a  descriptive  summary  of  these  data  is  presented  at 
Appendix  B.  The  summary  indicates  that  the  majority  of  students  participating 
in  the  experiment  received  their  commission  from  the  Reserve  Officer's 
Training  Corps  (ROTC)  and  were  serving  in  the  reserve  component,  either  the 
Army  Reserve  or  the  Army  National  Guard.  In  terms  of  military  experience,  the 
students  had  about  1.5-2  years  experience  prior  to  AOBC.  Finally,  the 
overwhelming  majority  of  students  (51  out  of  52)  reported  some  kind  of 
computer  experience.  Analyses  revealed  no  significant  differences  between 
experimental  and  control  groups  on  any  of  these  background  and  experience 
variables  except  in  one  case:  computer  experience  related  to  use  of  a 
computer  at  work.  In  that  particular  case,  a  higher  proportion  of  exper¬ 
imental  students  reported  having  computer  experience  at  work  compared  to 
control  students. 


Assessing  Effects  on  Tactical  Knowledges 

To  determine  whether  or  not  students  retained  factual  knowledges  learned 
from  ARTACT,  two  tactical  scenarios  were  prepared  for  a  tactical  knowledge 
test.  The  test  consisted  of  either  a  defensive  or  offensive  tactical 
scenario,  both  of  which  were  similar  in  content  to  exercises  presented  in  AOBC 
as  well  as  those  presented  in  ARTACT.  Student  responses  were  recorded  on 
audio  tape  and  were  later  transcribed  for  scoring.  Both  forms  of  the  test 
were  designed  to  be  administered  orally  on  an  individual  basis.  The  test 
began  with  a  recorded  company- level  0P0RD.  A  written  copy  of  the  0P0RD  and  a 
tactical  overlay  were  available  to  the  student  throughout  the  test.2  The 
experimenter  then  read  from  a  prepared  script  that  started  with  the  platoon  in 


2The  support  of  the  Platoon  Tactics  Branch  of  the  Armor  School's  Command 
and  Staff  Department  is  acknowledged  for  providing  the  0P0RD  and  overlays. 
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an  assembly  area,  moved  them  through  some  imaginary  events  to  which  the 
student  described  his  actions,  and  concluded  with  completion  of  the  mission. 

A  copy  of  the  OPORD,  tactical  overlay,  and  experimenter  instructions  for  both 
the  offensive  and  defensive  scenario  are  attached  at  Appendix  C.  Imbedded  in 
the  instructions  were  multiple  questions  designed  to  test  the  student's 
knowledge  of  tactics.  Included  in  the  instructions  (but  not  provided  to 
students)  are  the  responses  that  the  experimenters  deemed  "correct"  for 
scoring  purposes.  The  two  forms  of  the  test  were  designed  to  be  comparable  to 
the  extent  that  they  both  contained  an  approximately  equal  number  of  items 
that  addressed  the  following  eight  objectives:3 

1.  Given  a  company  mission  and  time  to  plan;  the  student  must  plan 
platoon  operations  in  accordance  with  standard  troop  leading  procedures 
including  analyzing  METT-T  factors  and  key  avenues  of  approach,  issuing  a 
warning  order,  and  executing  a  physical  reconnaissance. 

2.  Given  a  mission  requiring  movement  toward  some  objective;  the  student 
must  select  an  appropriate  route. 

3.  Given  a  route  to  the  objective  and  information  on  the  tactical 
situation;  the  student  must  select  an  appropriate  movement  technique  and 
movement  formation. 

4.  Given  a  movement  technique  and  formation;  the  student  must  recognize 
the  hand-and-arm  signals  corresponding  to  that  technique  and  formation. 

5.  Given  a  movement  formation;  the  student  must  specify  the  relative 
position  of  tanks  within  the  platoon  including  the  gun  tube  orientations  of 
each  tank. 

6.  Given  a  mission  requiring  field  artillery  support;  the  student  must 
formulate  an  appropriate  indirect  fire  plan  including  locating  preplanned 
targets,  selecting  symbols  for  each  target  (point,  area,  and  line),  and 
determining  shell/fuze  combinations. 

7.  Given  that  platoon  operations  are  underway;  the  student  must  react 
appropriately  to  surprise  situations  that  may  occur  including  encountering 
targets  at  and  beyond  direct  fire  range,  obstacles  such  as  streams  or 
minefields,  etc. 

8.  Given  a  situation  requiring  communication;  the  student  must  recognize 
when  to  authenticate  a  message,  when  to  execute  a  platoon  fire  command,  and 
when  to  provide  a  SPOTREP  or  SITREP. 


3The  sponsors,  developers,  and  evaluators  informally  agreed  that  these 
eight  statements  provided  a  fair  representation  of  ARTACT's  objectives. 
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Assessing  Effects  on  Organization  of  Tactical  Knowledges 


"Talk  Aloud"  Protocols 


The  instructions  for  the  tactical  knowledge  test  encouraged  students  to 
"talk  aloud"  as  they  answered  the  test  questions.  These  instructions  were 
modified  from  those  provided  by  Ericsson  and  Simon  (1984)  in  their  textbook  on 
protocol  analysis.  One  of  the  experimenters  performed  a  content  analysis  of 
student  protocols  revealing  ten  categories  of  student  comments  that  were 
incidental  to  their  answers  but  reflective  of  knowledge  organization  and 
processing.  All  transcripts  were  scored  by  this  experimenter  who  identified 
and  counted  the  number  of  instances  of  each  type  of  comment  while  remaining 
"blind"  with  respect  to  group  membership.  It  should  be  stressed  that  these 
comments  were  scored  independently  from  their  "correctness,"  which  is 
reflected  in  error  rates  on  the  tactical  knowledge  test.  Reliability  of 
scoring  scheme  was  examined  by  having  a  second  experimenter  score  six  of  the 
student  transcripts  independently  using  an  initial  set  of  instructions  for 
differentiating  the  categories. 

CHALLENGE 


CHALLENGE  is  an  experimental  computer  program  under  development  by  Kern 
and  Legree  (1988).  The  objective  of  the  program  is  to  assist  students  in 
organizing  their  knowledge  of  a  particular  topical  domain.  The  program  is 
designed  to  be  used  repeatedly  by  students  involved  in  a  course  of  instruc¬ 
tion.  However,  in  this  evaluation  it  was  used  only  during  pretest  and 
posttest  to  reflect  changes  in  scope  and  quality  of  students'  organization  of 
their  tactical  knowledge. 

The  CHALLENGE  program  presents  the  student  with  a  central  topic 
previously  entered  by  the  experimenter.  In  essence,  the  instructions  ask  the 
student  to  generate  and  edit  a  keyword  outline  to  represent  his  or  her 
understanding  of  the  central  topic.  In  this  evaluation,  the  central  topic  was 
"what  you  need  to  know  to  plan  an  armor  platoon  operation."  Summarizing  from 
Kern  and  Legree  (1988),  the  activities  presented  by  CHALLENGE  may  be  described 
by  three  sequential  phases  of  actions: 

1.  The  students  were  first  asked  to  list  the  major  concepts,  procedures, 
or  "things  you  need  to  know"  to  understand  the  central  topic.  The  current 
version  of  CHALLENGE  used  in  the  present  experiment  (CHALLENGE  II)  allows  the 
student  to  list  up  to  eight  different  major  topics.  However,  because  of  time 
limitations  in  the  present  research,  AOBC  students  were  instructed  to  generate 
a  maximum  of  five  such  major  topics. 

2.  The  students  were  then  asked  to  generate  "supporting  points"  for  each 
of  the  major  topics  presented  one  at  a  time.  Supporting  points  are 
subordinate  elements  that  the  student  considers  important  for  understanding 
each  major  topic.  Students  were  allowed  to  list  up  to  eight  different 
supporting  points  for  each  major  topic. 
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3.  The  final  process  required  the  student  to  rate  each  supporting  point 
in  terms  of  its  importance  to  understanding  each  of  the  major  topics.  The 
CHALLENGE  program  uses  these  data  to  generate  various  measures  that  tap  the 
interrelationships  between  major  topics. 

Assessing  Effects  on  Performance  in  AOBC 

The  intent  of  the  final  set  of  performance  data  was  to  determine  whether 
or  not  ARTACT  training  affected  performance  in  AOBC.  To  make  this 
determination,  academic  records  were  obtained  for  every  student  assigned  to 
the  experiment.  The  records  consisted  of  scores  on  academic  tests  given 
throughout  AOBC  and  students'  final  class  standings  expressed  in  percentile 
ranks.  In  addition,  records  of  student  performance  on  the  Armor  Performance 
Test  were  obtained.  This  particular  test  is  an  integrative  exercise  that  taps 
many  of  the  same  tactical  skills  and  knowledges  addressed  in  ARTACT.  The  test 
consists  of  two  components:  (a)  a  paper-and-pencil  knowledge  test  and,  (b)  a 
hands-on  performance  test  based  on  a  simulated  tactical  scenario  and 
administered  by  instructors  using  a  terrain  board.  In  the  interest  of  test 
security,  the  specific  items  on  the  test  cannot  be  revealed. 

Collecting  Formative  Data 

Since  the  present  experiment  represented  the  initial  implementation  of 
the  programs,  the  research  provided  an  opportunity  to  collect  formative 
evaluation  data  concerning  the  ARTACT  modules  and  associated  hardware  and 
software.  The  purpose  was  to  determine  how  well  the  software  and  hardware 
functioned  and  to  identify  software  and  hardware  changes  that  could  improve 
the  effectiveness  of  the  programs.  Two  types  of  data  were  obtained  for  this 
formative  evaluation:  (a)  a  record  of  software  and  hardware  problems  that 
occurred  during  the  implementation  of  the  four  ARTACT  modules,  and 
(b)  subjective  reactions  of  the  subjects  as  measured  on  a  questionnaire 
administered  at  the  end  of  the  study. 

Record  of  Software  and  Hardware  Problems 


During  the  research,  a  record  was  kept  of  all  problems  that  were 
experienced  during  the  implementation  of  ARTACT. 

AOBC/ARTACT  Questionnaire 

A  questionnaire  was  prepared  to  assess  subjective  reactions  to  ARTACT. 
The  first  five  items,  which  were  administered  to  students  in  both  the 
experimental  and  control  groups,  dealt  with  issues  related  to  AOBC.  The 
specific  subjects  of  these  items  and  predictions  concerning  the  expected 
differences  between  experimental  and  control  groups  are  described  below: 

1.  The  difficulty  levels  of  eleven  topics  presented  in  AOBC.  Assuming 
that  ARTACT  were  effective,  it  was  expected  that  the  AOBC  topics  that  were 
covered  in  ARTACT  would  be  perceived  to  be  less  difficult  by  students  in  the 
experimental  group  than  by  students  in  the  control  group.  No  differences 
between  the  two  groups  were  expected  for  topics  that  were  not  covered  in 
ARTACT. 
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2.  Interest  in  AOBC  instruction  on  tactics.  Assuming  again  that  ARTACT 
were  effective,  interest  in  AOBC  tactics  instruction  was  expected  to  be  higher 
among  students  in  the  experimental  group. 

3.  Preparedness  for  the  10-day  field  exercise  presented  at  the  end  of 
AOBC.  On  the  assumption  that  ARTACT  were  effective,  it  was  expected  that  the 
students  in  the  experimental  group  would  benefit  more  from  their  AOBC 
instruction  and  would  perceive  themselves  to  be  better  prepared  for  the  10-day 
field  exercise  given  at  the  end  of  AOBC. 

4.  Knowledge  of  planning  and  executing  platoon  tactics.  It  was 
anticipated  that  as  a  result  of  their  ARTACT  participation,  the  students  in 
the  experimental  group  would  perceive  themselves  to  be  more  knowledgeable  in 
planning  and  executing  platoon  tactics. 

5.  Preparedness  to  serve  in  combat  as  the  leader  of  a  tank  platoon.  It 
was  anticipated  that  the  students  in  the  experimental  group  would  perceive 
themselves  to  be  better  prepared  to  serve  as  a  platoon  leader  in  combat. 

Ten  additional  items,  which  were  administered  only  to  students  in  the 
experimental  group,  assessed  their  reactions  to  ARTACT.  Specifically,  these 
10  items  dealt  with  the  following  ARTACT  issues:  (a)  student  perceptions  of 
ARTACT,  (b)  their  suggestions  for  program  improvements,  and  (c)  their 
recommendations  concerning  the  role  that  ARTACT  could  perform  in  AOBC.  A  copy 
of  the  experimental  group  questionnaire,  which  includes  all  15  questions 
items,  is  attached  at  Appendix  A-2. 

Procedure 


The  experimental  procedure  was  organized  around  three  major  events: 

(a)  the  pretest,  (b)  the  ARTACT  training  sessions,  and  (c)  the  posttest.  Each 
of  these  three  events  is  described  in  detail  below. 

Pretest 


Students  in  both  experimental  and  control  groups  were  initially  tested  in 
Week  4  of  AOBC.  Three  types  of  data  were  obtained  at  that  time.  The  first  of 
these  related  to  background  information  on  the  students.  Students  provided 
these  data  in  response  to  a  written  questionnaire.  (See  Appendix  A-l.) 
Following  the  questionnaire,  students  interacted  with  the  CHALLENGE  program  to 
determine  their  initial  knowledge  organization  for  planning  a  platoon 
operation.  Student  output  from  CHALLENGE  was  saved  on  hardcopy  printouts  and 
on  floppy  diskettes.  After  CHALLENGE,  the  experimenters  administered  the 
tactical  knowledge  test  on  an  individual  basis  while  tape  recording  student 
responses.  Approximately  half  of  the  students  received  the  offensive  scenario 
while  the  other  half  received  the  defensive  scenario.  The  entire  pretest 
lasted  approximately  1.5-2  hours. 
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ARTACT  Training  Sessions 

Students  received  the  training  in  individual  sessions  scheduled  for  up  to 
two  hours  each.  Two  student  work  stations  were  available  for  conducting  this 
training  and  were  located  in  the  Armor  School's  Professional  Development 
Center/  Table  1  shows  the  scheduling  planned  for  the  four  modules  of  ARTACT 
versus  the  corresponding  blocks  of  AOBC  instruction. 


Table  1 

Correspondence  Between  Planned  Schedules  of  ARTACT  Modules  and  AOBC  Classes 


ARTACT  Modules 

AOBC  Classes 

Topic  Week 

of  AOBC 

Topic  Week 

of  AOBC 

Planning  FA  Fires 

5th 

Platoon  Fire  Plans 

14th 

Techniques  of  Movement 

7th 

Techniques  of  Movement 

12th 

Offensive  Operations 

12th 

Fundamentals  of  Offense 

13th 

Defensive  Operations 

13th 

Fundamentals  of  Defense 

14th 

Student  time  for  working  on  these  programs  could  not  conflict  with 
scheduled  AOBC  instruction  and  had  to  come  from  the  students 's  personal  time, 
such  as  evenings  and  time  off  during  weekends.  As  a  result,  the  week  that  a 
student  actually  received  a  particular  ARTACT  program  sometimes  deviated  from 
the  planned  schedule.  Missed  appointments  and  system  failures  (in  the  first 
two  weeks)  were  responsible  for  most  of  the  deviations.  The  distribution  of 
actual  times  that  students  received  ARTACT  training  is  shown  in  Table  2. 

Student  time  to  complete  each  of  the  four  ARTACT  modules  is  shown  in 
Table  3.  Although  the  sessions  were  monitored  by  a  member  of  the  research 
staff,  all  of  the  instruction  was  presented  by  the  ARTACT  system.  Planning 
Field  Artillery  (FA)  Fires  was  the  shortest  module  and  the  only  one  that 
students  worked  through  twice  within  the  same  session. 


4Each  work  station  consisted  of  a  Zenith  248  computer,  a  Pioneer  LDV 
6010A  laser  videodisc  player,  two  Sony  color  video  monitors  (model  PVM-1271Q), 
an  ALPS  dot  matrix  printer,  and  an  audio  headset. 
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Table  2 

Distribution  of  Actual  Times  Students  Were  Trained  on  ARTACT  Modules 


_ Week  of  AOBC _ 

ARTACT  Module  5th  6th  7th  8th  9th  10th  Uth  12th  13th  14th 


Planning 

FA  Fires  15  2  4  5  2 

Techniques  of 

Movement  17  0  7  1  3 

Offensive 

Operations  3  8  16  1 

Defensive 

Operations  16  12 


Table  3 

Student  Time  to  Complete  ARTACT  Modules 


Time  (in  mins) 


ARTACT  Module 

MON 

RNG 

Planning  FA  Fires  (Once) 

20 

12-35 

Techniques  of  Movement 

55 

35-110 

Offensive  Operations 

45 

30-70 

Defensive  Operations 

55 

40-95 
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Posttest 


Students  in  both  groups  were  tested  for  a  second  time  in  Week  16. 

Students  were  asked  to  complete  three  activities.  The  first  activity  was  to 
retake  the  CHALLENGE  program  under  exactly  the  same  instructions  used  in  the 
pretest.  The  second  activity  was  to  take  the  alternate  version  of  the 
tactical  test.  That  is,  students  who  received  the  offensive  tactical  scenario 
on  the  pretest  were  assigned  the  defensive  scenario  for  the  posttest  (test 
order  A),  whereas  those  whose  received  the  defensive  scenario  on  the  pretest 
were  given  the  offensive  scenario  on  the  posttest  (test  order  B).  The  third 
posttest  activity  required  the  students  to  fill  out  a  questionnaire  that 
addressed  their  perceptions  of  AOBC.  The  experimental  students  were  also 
given  additional  questions  relating  to  their  perceptions  of  ARTACT.  Like  the 
pretest,  the  entire  posttest  session  lasted  1.5  -  2  hours. 

Design  and  Analysis 

In  the  analysis  of  variance  (ANOVA)  design  used  to  analyze  performance 
data,  students  were  nested  into  either  control  or  experimental  conditions 
(groups)  and  crossed  with  repeated  test  administrations  (trials).  Variations 
and  elaborations  to  this  basic  design  are  noted  in  the  relevant  sections  of 
the  Results.  Because  ARTACT  was  administered  between  the  pretest  and  the 
posttest,  the  effects  of  ARTACT  were  evaluated  by  the  group  X  trials 
interaction  term.  Given  a  significant  interaction,  simple  effects  analyses 
were  performed  using  the  within-subject  error  term  to  determine  the 
differences  between  pretest  and  posttest  within  one  of  the  two  groups.  To 
test  differences  between  groups  either  at  the  pretest  or  at  the  posttest,  the 
error  term  was  pooled  between-subjects  and  within-subjects  yielding  a  "within 
cell"  estimate  of  variance.  The  significance  of  this  effect  was  estimated 
using  a  ^approximation  method  attributed  to  W.  G.  Cochran  and  described  in 
Lindquist  (1953).  For  analyses  involving  more  than  two  dependent  measures, 
univariate  ANOVAs  were  supplemented  with  multivariate  analyses  of  variance 
(MANOVAs)  to  protect  against  violations  of  the  compound  symmetry  assumption  in 
univariate  ANOVAs.  Following  advice  from  Wilkinson  (1986),  if  the  results 
from  the  two  analyses  disagreed,  the  decision  from  the  MANOVA  took  precedence 
because  it  does  not  require  this  assumption. 
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Results  and  Discussion 


The  results  of  the  evaluation  are  organized  into  sections  corresponding 
to  individual  evaluation  issues.  The  sections  begin  with  details  on  data 
scoring  procedures,  if  applicable.  The  first  four  sections  present  the 
results  from  four  groups  of  performance  measures,  all  of  which  measure 
different  aspects  of  the  students'  knowledge  of  tactics.  To  determine  the 
relationships  between  these  performance  measures,  one  or  two  of  the  most 
representative  variables  from  each  of  these  four  groups  of  performance 
variables  were  selected  for  a  correlational  analysis,  the  results  of  which  are 
presented  in  Appendix  D.  The  intercorrelations  calculated  over  both  the 
experimental  and  control  groups  are  presented  in  Table  D-l  with  correlations 
within  these  groups  presented  in  Tables  D-2  and  D-3  respectively.  Findings 
from  this  correlational  analysis  are  discussed  in  the  appropriate  results 
sections  along  with  the  findings  of  other  analyses  of  the  data.  Each  section 
concludes  with  a  short  summary  of  the  results  of  the  analyses. 

Effects  on  Tactical  Knowledges: 

Errors  on  the  Tactical  Knowledge  Test 

Data  Scoring  Procedures 

Tape  recordings  of  student  responses  to  the  tactical  knowledge  test  were 
transcribed  and  then  scored  for  accuracy.  Technical  problems  resulted  in  the 
failure  to  record  the  posttest  responses  of  one  student  in  the  control  group. 
His  data  were  excluded  from  these  analyses  resulting  in  ns  of  25  and  26  for 
experimental  and  control  groups,  respectively.  For  each  student,  an  error 
rate  was  calculated  for  each  of  the  eight  test  objectives  covered  in  the 
tactical  scenarios.  The  error  rate  was  defined  as  the  number  of  errors 
committed  divided  by  the  total  number  of  scoreable  items  for  that  particular 
objective.  Because  the  final  two  objectives  (Surprise  Events  and  Reports) 
were  based  on  only  four  and  two  items  respectively,  performance  in  some  of  the 
experimental  subgroups  had  no  variance.  Consequently,  results  from  both 
objectives  were  combined  to  make  a  single  objective  for  purposes  of  analysis. 
The  design  used  to  analyze  these  data  set  students  nested  into  test  orders 
(A  or  B)  as  well  as  into  groups  forming  a  between-groups  2X2  factorial 
design  (group  X  test  order).  Students  were  also  crossed  with  the  resulting 
seven  test  objectives  and  the  two  test  administrations. 

Results  of  the  Analyses 

Effects  on  overall  performance.  Table  4  presents  mean  error  rates  over 
all  test  objectives  of  the  tactical  knowledge  test  for  experimental  and 
control  groups.  The  most  obvious  trend  in  these  data  is  that  students  in  both 
groups  committed  substantially  fewer  errors  on  the  posttest  than  on  the 
pretest.  Analyses  of  these  data  (summarized  in  Appendix  Table  E-l)  confirmed 
that  these  trends  were  highly  significant,  F  (1,  47)  «  344.34,  p  <  .001. 
Furthermore,  decreases  in  error  rate  were  significant  for  both  experimental 
and  control  groups,  fs  (1,  27)  =  126.43  and  226.35,  respectively,  both 
ps  <  .001.  However,  the  experimental  group  showed  a  greater  decrease  in 
errors  from  pretest  to  posttest  than  did  the  control  group  as  indicated  by  the 
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Table  4 


Means  and  Standard  Deviations  of  Overall  Error  Rates  on  the  Tactical 
Knowledge  Test 


Group 

As  Measured  at 

Pretest  Posttest 

Experimental 

M 

0.66 

0.29 

SD 

0.11 

0.07 

Control 

M 

0.61 

0.34 

SD 

0.11 

0.10 

significant  group  X  trial  interaction,  F  (1,  47)  =  8.16,  p  <  .01.  As  can  be 
seen  in  Table  4,  the  interaction  was  disordinal  in  that  the  relationship 
between  experimental  and  control  groups  was  reversed  from  pre-  to  posttest. 

As  a  result  of  this  disordinal  relationship,  the  differences  between  groups  at 
the  pretest  and  at  the  posttest  were  small  and  nonsignificant  (both  ps  >  .10). 
Nevertheless,  the  significant  interaction  indicated  that  improvement  in 
performance  between  pretest  and  posttest  was  greater  for  the  experimental  than 
for  the  control  group. 

Effects  on  performance  on  individual  test  objectives.  Table  5  allows  an 
examination  of  these  same  trends  by  individual  test  objectives.  The  table 
shows  large  differences  in  error  rates  among  the  objectives,  which  were  shown 
to  be  statistically  significant,  F  (6,  282)  =  16.00,  p  <  .001.  Furthermore, 
there  were  differences  in  performance  gains  among  the  objectives.  That  is, 
performance  on  some  objectives  (e.g.,  hand-and-arm  signals)  improved  more  than 
performance  on  other  objectives  (e.g.,  indirect  fire  planning).  The 
corresponding  AN0VA  effect  (objective  X  trial)  was  confirmed  to  be  signifi¬ 
cant,  F  (6,  282)  =  18.84,  p  <  .001.  Despite  the  differences  in  performance 
gains  among  objectives,  the  table  shows  that  the  trend  for  greater  performance 
improvement  in  experimental  than  in  control  students  is  evident  on  every 
objective.  This  observation  is  supported  by  the  fact  that  the  group  X 
objective  X  trial  interaction  was  not  significant,  F  <  1.  Thus,  the  analyses 
support  the  contention  that  the  effect  of  ARTACT  was  uniform  across  training 
objectives. 
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Table  5 


Means  and  Standard  Deviations  of  Error  Rates  on  Individual  Objectives 
of  the  Tactical  Knowledge  Test 


As  Measured  at 

Test  Objective 

Group 

Pretest 

Posttest 

Troop  Leading  Procedures 

Experimental 

M 

0.72 

0.34 

SD 

0.20 

0.34 

Control 

M 

0.72 

0.36 

SD 

0.21 

0.30 

Route  Selection 

Experimental 

M 

0.49 

0.35 

SD 

0.17 

0.19 

Control 

M 

0.47 

0.33 

SD 

0.15 

0.19 

Movement  Technique/Formation 

Experimental 

M 

0.71 

0.46 

SD 

0.23 

0.16 

Control 

M 

0.55 

0.49 

SD 

0.19 

0.24 

Hand-and-Arm  Signals 

Experimental 

M 

0.70 

SD 

0.27 

Control 

M 

0.58 

SD 

0.27 

Formation  Drawing 

Experimental 

M 

0.70 

0.16 

SD 

0.24 

0.12 

Control 

M 

0.58 

0.18 

SD 

0.25 

0.17 

Indirect  Fire  Planning 

Experimental 

M 

0.54 

0.36 

SD 

0.21 

0.17 

Control 

M 

0.56 

0.49 

SD 

0.21 

0.18 

Surprise  Events/Reports 

Experimental 

M 

0.73 

0.31 

SD 

0.23 

0.24 

Control 

M 

0.77 

SD 

0.17 

0.24 
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Test  order  was  included  in  these  analyses  as  a  control  variable.  The 
main  effect  of  test  order  was  not  significant  nor  were  any  of  the  first  order 
interactions  with  order.  The  only  interaction  with  test  order  that  was 
significant  according  to  both  univariate  and  multivariate  criteria  was  the 
triple  order  X  objective  X  trial  interaction,  F  (6,  282)  =  5.84,  p  <  .001.  To 
explain  this  interaction,  the  performance  data  were  broken  down  by  test  order 
(A  or  B),  test  objective,  and  trial  and  are  presented  in  Table  6.  The  mean 
values  indicate  that  there  were  differences  between  the  offensive  and 
defensive  scenarios  on  individual  objectives.  For  instance,  the  data 
indicated  more  errors  on  the  defensive  scenario  than  on  the  offensive  scenario 
for  route  selection,  whereas  this  relationship  is  reversed  for  surprise 
events/reports.  Although  there  were  minor  differences  between  test  versions, 
the  lack  of  interaction  between  test  order  and  group  membership  suggests  that 
test  order  had  no  mitigating  effect  on  any  of  the  previous  findings. 


Table  6 

Mean  Error  Rates  on  Tactical  Knowledge  Test  by  Objective,  Test  Order, 
and  Trial 


Test  Objective 

Test  Order* 

As  Measured  at 

Pretest  Posttest 

Troop  Leading  Procedures 

A 

0.73 

0.40 

B 

0.71 

0.29 

Route  Selection 

A 

0.47 

0.39 

B 

0.49 

0.28 

Movement  Technique/ 

A 

0.64 

0.40 

Formation 

B 

0.62 

0.56 

Hand  &  Arm  Signals 

A 

0.60 

0.06 

B 

0.67 

0.12 

Formation  Drawing 

A 

0.58 

0.14 

B 

0.70 

0.20 

Indirect  Fire  Planning 

A 

0.51 

0.44 

B 

0.61 

0.41 

Surprise  Events/Reports 

A 

0.82 

0.26 

B 

0.67 

0.47 

‘Test  order  A  denotes  that  students  took  the  offensive  version  of  the 
tactical  knowledge  on  the  pretest  and  the  defensive  version  on  the 
posttest.  Test  order  B  indicates  that  the  student  took  the  test  versions 
in  the  opposite  order. 
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As  revealed  in  the  Method  section  (Table  2),  scheduling  problems 
resulted  in  some  students  not  receiving  ARTACT  at  the  planned  time  with 
respect  to  corresponding  units  of  instruction.  Post  hoc  analyses  were 
conducted  to  contrast  the  performance  of  students  who  received  ARTACT  as 
planned  versus  those  who  received  ARTACT  after  the  planned  time.  Results  of 
these  analyses  showed  no  differences  in  performance  on  the  tactical  knowledge 
test  as  well  as  other  variables.®  These  findings  suggest  that  ARTACT's 
effects  were  not  dependent  on  strict  adherence  to  the  planned  schedule. 

Correlations  with  other  knowledge  variables.  Appendix  D  indicates  that 
there  were  more  significant  bivariate  correlations  involving  the  tactical 
knowledge  test  than  any  of  the  other  knowledge  measures.  Of  the  eighteen 
correlations  involving  the  pre-  and  posttest  administrations  of  the  tactical 
knowledge  test,  six  were  significant  at  the  .05  level  and  three  more 
approached  significance--! .e. ,  ps  <  .10.  The  correlations  of  performance  on 
this  variable  to  other  knowledge  variables  provide  evidence  for  its  validity 
as  a  measure  of  tactical  knowledge. 

Summary 

The  findings  from  these  analyses  may  be  summarized  by  three  generali¬ 
zations: 


1.  There  were  large  decreases  in  error  rate  between  pretest  and  posttest 
in  both  experimental  and  control  groups.  In  view  of  the  fact  that  students 
took  a  different  form  of  the  tactical  knowledge  test  at  the  posttest,  it  is 
unlikely  that  all  or  even  most  of  this  large  performance  improvement  was  due 
to  repeated  testing.  It  is  more  likely  that  most  of  this  improvement  was  due 
to  the  students'  intervening  experience,  i.e.,  their  AOBC  course  work. 

2.  Students  in  the  ARTACT  group  improved  more  than  those  in  the  control 
group.  This  finding  supported  the  hypothesis  that  students  learn  and  apply 
the  subset  of  AOBC  knowledges  tutored  in  ARTACT. 

3.  The  greater  improvement  for  the  ARTACT  group  was  uniform  across  all 
of  the  seven  performance  objectives. 

Effects  on  Organization  of  Tactical  Knowledge: 

1.  Talk  Aloud  Protocols 


Data  Scoring  Procedures 

As  a  result  of  the  instructions  to  "talk  aloud,"  students  provided 
verbalizations  that  were  incidental  to  the  questions  posed  in  the  tactical 
knowledge  test.  The  analysis  of  these  additional  verbal  data  was  begun  by 
performing  a  content  analysis  of  student  verbalizations  from  the  first  AOBC 
class  whose  data  were  not  included  in  the  results.  Ten  categories  or  types  of 


details  on  these  analyses  may  be  obtained  from  the  authors. 
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incidental  verbalizations  were  identified  along  with  examples  of  each  from 
actual  student  transcripts.  Types  of  verbalizations  included  elaborations  on 
how  procedures  are  performed  as  well  as  reasons  for  choosing  particular 
responses.  Appendix  F  defines  each  of  these  ten  categories  and  provides 
verbatim  examples  of  each  type  from  student  transcripts.  After  establishing 
these  categories  of  responses,  transcripts  from  the  remaining  five  classes 
were  read  to  identify  every  occurrence  of  these  verbalizations.  The  mean 
number  of  occurrences  of  each  type  of  comment  was  tabulated  for  each  student. 
The  mean  number  of  comments  is  displayed  in  Appendix  Table  F-l.  In  decreasing 
frequency  of  occurrence,  the  ten  categories  of  comments  were: 

1.  Reasons  related  to  specific  characteristics  of  the  situation  or  of 
the  prescribed  action. 

2.  Reasons  related  to  predicted  outcomes  of  prescribed  actions. 

3.  General  principles  that  guide  responses. 

4.  Reasons  related  to  information  in  the  original  OPORD  or  FRAGO. 

5.  Reasons  related  to  anticipated  enemy  actions. 

6.  Procedural  information  on  "how  to11  perform  a  task. 

7.  Reasons  related  to  the  students  interpretation  of  the  mission. 

8.  Reasons  based  solely  on  doctrine  or  SOP  or  some  other  rote  tactical 

rule. 

9.  Reasons  based  on  personal  preference. 

10.  Evidence  of  some  mnemonic  or  imaginal  coding. 

These  categories  of  incidental  verbalizations  were  examined  as  possible 
indicants  of  the  types  of  knowledge  that  students  used  to  answer  questions  on 
the  tactical  knowledge  test.  Upon  examination,  it  was  determined  that  the  two 
least  frequent  categories  (personal  preference  and  mnemonic  coding)  had  no 
clear  relation  to  knowledge  and  were  therefore  not  considered  further.  Of  the 
remaining  eight  categories,  six  constituted  statements  of  the  students' 
reasons  for  their  answers  to  questions  on  the  tactical  knowledge  test,  while 
two  did  not  reflect  reasons  per  se,  but  were  indicative  of  knowledge 
development. 

Classification  of  "reasons-for-answers-qiven"  categories.  The  six 
categories  reflecting  reasons  for  answers  given  were  recombined  according  to 
the  "level"  of  knowledge  that  they  were  determined  to  represent  (Perkins  & 
Salomon,  1989).  At  the  lowest  or  most  specific  level  of  knowledge  were 
reasons  based  on  the  local  or  proximal  conditions  that  control  performance. 
This  level  of  knowledge  is  specific  in  the  sanse  that  it  is  applicable  only  to 
the  domain  in  question  and  is  therefore  characteristic  of  the  expert 
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performer.  The  response  categories  corresponding  to  this  level  of  knowledge 
were  reasons  based  on  specific  characteristics  and  information  provided  in  the 
0P0RD/FRAG0  (categories  1  and  4).  This  level  is  labeled  Specific  Knowledge  in 
Table  7. 

Two  categories  (categories  7  and  8)  contain  reasons  based  on  the 
student's  interpretation  of  the  mission  or  on  doctrine  and  SOP.  These  reasons 
appear  to  reflect  an  intermediate  level  of  knowledge  in  that  they  are  domain- 
specific,  but  based  on  abstractions  rather  than  on  local  conditions.  This 
level  is  labeled  Intermediate  Level  in  Table  7. 

The  remaining  two  of  the  six  "reasons-for-answers-given"  categories 
(categories  2  and  5)  were  classified  as  knowledges  at  a  higher  level  of 
generality,  reflecting  heuristics  applicable  to  tactics  as  well  as  to  other 
knowledge  domains.  General  heuristics  are  characteristic  of  the  sophisticated 
novice.  The  term  "sophisticated"  is  meant  to  imply  that,  in  order  to  use 
general  heuristics,  the  student  must  recognize  the  applicability  of  what  he 
already  knows  to  a  new  domain--some\:hing  that  students  do  not  necessarily  do. 
Responses  classified  into  this  level  of  knowledge  were  reasons  based  on  the 
predicted  outcomes  of  a  response  or  on  the  anticipated  actions  of  the  enemy. 
This  level  is  labeled  General  Heuristics  in  Table  7. 

The  numbers  of  comments  characteristic  of  each  of  three  levels  of 
knowledge  were  tabulated  for  each  student.  The  number  of  comments  within  each 
level  was  analyzed  separately  setting  students  nested  into  groups  and  crossed 
with  repeated  test  administrations  (trials). 

Comments  not  presented  as  reasons  for  answers  given.  The  two  remaining 
categories  of  incidental  comments  (categories  3  and  6)  were  not  reasons  per 
se,  but  they  were  indicative  of  knowledge  development.  The  first  category 
(Table  8)  is  where  the  student  offered  some  statement  of  general  principles 
that  could  be  used  to  guide  responses  but  were  not  used  to  justify  any 
specific  response.  For  instance,  one  student  said  that  his  choice  of  movement 
technique  would  be  based  on  the  likelihood  of  enemy  engagement,  but  he  did  not 
indicate  how  the  choice  was  actually  made.  These  sorts  of  responses  may  be 
examples  of  declarative  knowledges  (facts)  that  have  not  been  compiled  into 
useable  procedures  (Anderson,  1982)  and  are  typical  of  the  early  stages  of 
learning. 

The  remaining  category  of  comments  is  where  the  student  provided 
unsolicited  procedural  information  on  how  to  perform  a  particular  task 
(Table  8).  For  instance,  when  asked  what  to  do  when  faced  with  direct  fire 
from  identifiable  enemy  tanks,  some  soldiers  not  only  said  that  they  would 
issue  a  platoon  fire  command,  they  also  described  the  content  of  that  fire 
command.  This  sort  of  response  may  be  the  converse  of  previous  type  of 
comment  in  that  the  student  demonstrates  elaborate  knowledge  of  procedure.  As 
in  the  previous  case,  data  in  these  two  categories  were  analyzed  separately. 
Due  to  the  low  frequency  of  comments  within  these  categories,  these  data  did 
not  meet  the  normality  and  homogeneity  of  variance  requirements  of  parametric 
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analyses.  Consequently,  soldiers  in  both  groups  were  classified  as  either 
having  increased  in  the  number  of  comments,  decreased  in  number,  or  showed  no 
change  across  the  repeated  test  administrations.  These  frequency  data  were 
then  analyzed  using  chi-square  techniques. 

Scorer  reliability.  To  check  on  the  reliability  and  objectivity  of  the 
scoring  of  the  incidental  verbalizations,  two  experimenters  independently 
scored  the  same  six  transcripts.  The  correlations  between  experimenters  on 
the  number  of  instances  across  the  five  categories  in  the  revised  scheme 
ranged  from  0.53  to  0.99+  with  an  average  0.78.  Detailed  analysis  of  the 
differences  between  scorers  revealed  two  types  of  discrepancies:  (a)  failures 
of  one  or  the  other  experimenter  to  recognize  instances  of  certain  types  of 
comments,  and  (b)  disagreements  in  classifications  of  comments.  The  first 
type  of  discrepancy  was  reduced  over  repeated  cycles  of  scoring  and  review, 
i.e.,  with  practice.  The  second  type  of  discrepancy  was  reduced  by 
clarifications  to  the  definitions  of  the  categories.  Although  a  perfect  1.0 
correlation  between  scorers  is  probably  not  possible  given  the  difficulties 
noted  by  the  experimenters  in  interpreting  student  transcripts,  the  findings 
suggest  that  higher  levels  of  agreement  could  have  been  achieved.  On  the 
basis  of  these  findings,  the  scoring  method  was  deemed  adequately  reliable  and 
objective. 

Results  of  the  Analyses 

Effects  on  generality  of  problem-solving  reasoning.  Table  7  presents 
the  mean  frequency  of  comments  corresponding  to  the  knowledge  levels  discussed 
above.  The  most  notable  trend  in  these  data  are  the  changes  in  the  frequency 
of  occurrence  across  repeated  test  administrations:  Whereas  the  comments 
indicative  of  specific  knowledge  increase  across  the  test  trials,  the  comments 
related  to  general  heuristics  decrease.  Analyses  of  these  data  (Appendix 
Table  E-2)  indicated  that  both  of  these  changes  across  trials  were  statisti¬ 
cally  reliable,  Fs  (1,  49)  =  4.08  and  4.37,  respectively,  both  ps  <  .05.  The 
increase  in  specific  knowledge  comments  would  confirm  the  students'  increasing 
expertise  in  tactics.  The  decrease  in  general  heuristics  comments  also 
reflects  increasing  expertise  in  accordance  with  the  hypothesis  offered  by 
Alexander  and  Judy  (1988):  "...  as  the  learners'  knowledge  of  the  content 

relative  to  that  task  increases,  then  it  is  likely  that  the  need  for  strategic 
behavior  [i.e.,  use  of  general  heuristics]  decreases"  (p.  375).  Comments 
relating  to  the  intermediate  level  of  knowledge  were  relatively  less  frequent 
than  the  other  two  categories  with  little  change  across  trials.  Analysis  of 
this  level  indicated  that  the  difference  between  trials  was  not  significant, 

F  <  1.  The  second  trend  notable  in  Table  7  is  that  the  changes  in  comments 
related  to  specific  knowledges  and  general  heuristics  were  more  pronounced  in 
the  experimental  group  than  in  the  control  group.  However,  the  analyses  did 
not  confirm  this  observation:  The  group  X  trial  interaction  was  not 
significant  in  any  of  the  three  analyses. 
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Table  7 

Means  and  Standard  Deviations  of  the  Frequency  of  Student  Comments 
Associated  With  Three  Levels  of  Knowledge  Generality 


As  Tested  at 

Knowledge  Level  Group  Pretest  Posttest 


Specific  Knowledge 

Experimental 

M 

3.72 

4.92 

SD 

1.84 

2.40 

Control 

M 

4.62 

4.92 

SD 

2.06 

2.73 

Intermediate  Level 

Experimental 

M 

0.32 

0.40 

SD 

0.56 

0.71 

Control 

M 

0.35 

0.42 

SD 

0.74 

0.90 

General  Heuristics 

Experimental 

M 

4.00 

3.08 

SD 

3.57 

1.60 

Control 

M 

3.73 

3.08 

SD 

2.31 

2.19 

Effects  on  types  of  incidental  knowledges.  Table  8  displays  the 
frequency  and  percent  of  students  in  each  group  classified  as  either 
increasing,  not  changing,  or  decreasing  in  the  number  of  comments  related  to 

General  principles  or  procedural  information.  An  equal  proportion  of  students 
about  35%  overall)  increased  as  decreased  in  their  comments  related  to 
general  principles.  Furthermore,  the  table  shows  more  experimental  students 
increasing  than  decreasing  in  frequency  of  comments,  whereas  the  control  group 
shows  the  opposite  tendency.  However,  a  chi-square  analysis  of  these  data 
indicates  that  the  group  differences  were  not  significant,  Chi2  (2)  =  2.80, 
p  =  .25.  In  contrast,  nearly  40%  of  the  students  evidenced  an  increase  in 
comments  related  to  procedural  information  with  no  students  showing  decreases. 
This  tendency  is  slightly  more  pronounced  in  the  control  group.  However, 
analysis  of  cells  with  non-zero  observed  frequencies  indicated  that  the  group 
differences  were  not  significant,  Chi1  (1)  =  0.21,  p  -  .65. 
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Table  8 


Frequency  and  Percent  of  Soldiers  in  Groups  Classified  by  Their  Change  in 
Number  of  Conments  from  Pre-  to  Posttest 


Change  in  Number  of 
Comments  from  Pre-  to  Posttest 


Comment 

Increase 

No  Change 

Decrease 

General  Principles 

Experimental 

f 

10 

9 

6 

% 

40 

36 

24 

Control 

f 

8 

6 

12 

% 

31 

23 

46 

Procedural  Information 

Experimental 

f 

9 

16 

0 

% 

36 

64 

0 

Control 

f 

11 

16 

0 

% 

41 

59 

0 

Correlations  with  other  knowledge  variables.  Appendix  D  presents  the 
correlations  of  the  frequency  of  comments  related  to  specific  knowledge  and 
general  heuristics  to  each  other  and  to  other  performance  measures.  The 
previous  analyses  of  means  clearly  indicated  that  these  two  types  of  comments 
were,  to  some  degree,  independent  as  evidenced  by  the  change  in  their  relative 
position  as  a  function  of  trials.  Nevertheless,  the  correlation  analysis 
indicated  that  these  measures  were  positively  related  to  each  other  at  the 
pretest  and  the  posttest,  although  the  correlation  at  the  posttest  only 
approached  significance,  p  <.07.  Another  notable  observation  from  this 
analysis  is  that  the  incidence  of  these  comments  at  the  pretest  was  correlated 
with  the  incidence  at  the  posttest.  A  plausible  explanation  of  these 
correlations  is  that  students  differ  with  respect  to  their  willingness  to 
"talk  aloud"  and  that  those  who  provided  one  sort  of  comment  were  likely  to 
provide  a  comment  of  the  other  sort  as  well.  These  data  can  also  be 
interpreted  as  supporting  the  generalization  offered  by  Alexander  and  Judy 
(1988)  that  specific  knowledge  and  general  heuristics  are  inextricably  related 
in  at  least  two  ways:  (a)  some  minimal  level  of  specific  knowledge  must  be 
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achieved  to  use  general  heuristics,  and  (b)  general  heuristics  can  improve  the 
use  of  specific  knowledge.  Finally,  perhaps  the  most  interesting  finding  from 
this  analysis  is  that  the  incidence  of  both  types  of  comments  were  negatively 
related  to  the  error  rate  on  the  tactical  knowledge  test,  i.e.,  higher 
incidence  of  comments  were  associated  with  lower  error  rates.  These  correla¬ 
tions  were  significant  at  the  pretest  (both  ps  <.01 ),  but  only  approached 
significance  at  the  posttest,  ps  <  .10.  Unfortunately,  the  present  correla¬ 
tional  data  cannot  be  used  to  determine  whether  use  of  specific  knowledge  and 
general  heuristics  strategies  caused  better  performance  on  the  knowledge  test, 
or  whether  these  verbalizations  and  test  performance  were  correlated  with  some 
enduring  student  trait  (e.g.,  intelligence).  Nevertheless,  these  findings 
suggest  that  use  of  knowledge  at  either  the  specific  or  general  level  is 
associated  with  good  performance,  a  finding  which  is  consistent  with  the 
literature  (Alexander  &  Judy,  1988). 

Summary 

Two  major  generalizations  may  be  drawn  from  these  data: 

1.  The  changes  in  types  of  comments  from  pretest  to  posttest  were 
generally  indicative  of  increasing  tactical  expertise.  In  particular,  the 
increase  in  problem-solving  reasons  based  on  specific  knowledge  and  the 
increase  in  procedural  knowledge  support  this  trend.  In  addition,  the 
decrease  in  use  of  general  heuristics  may  indicate  that  students  relied  less 
on  strategies  indicative  of  the  novice  performer  as  their  expertise  developed. 

2.  These  findings  failed  to  show  greater  changes  in  the  experimental  as 
opposed  to  the  control  group.  That  these  trends  were  evident  in  the  control 
as  well  as  the  experimental  group  implied  that  the  primary  cause  of  these 
shifts  in  types  of  comments  was  AOBC  instruction  and  not  ARTACT  tutoring. 

Effects  on  Organization  of  Tactical  Knowledge: 

2.  The  CHALLENGE  Program 


Data  Scoring  Procedures 

Two  types  of  data  were  obtained  from  CHALLENGE.  The  first  category  of 
data  related  to  the  student's  confidence  in  the  results  of  CHALLENGE.  As  a 
last  step  in  the  CHALLENGE  program,  students  rated  their  confidence  in  the 
central  topic  on  a  100-point  scale.  As  stated  earlier,  the  central  topic  for 
CHALLENGE  was  "what  you  need  to  know  to  plan  an  armor  platoon  operation."  Due 
to  technical  problems,  the  pretest  confidence  ratings  of  one  student  who  was 
assigned  to  the  control  group  were  lost.  Consequently,  his  data  were  excluded 
from  the  following  analyses  resulting  in  ns  of  25  and  26  for  the  experimental 
and  control  groups,  respectively.  The  remaining  CHALLENGE  data  are  based  on 
ns  of  25  and  27  for  experimental  and  control  groups. 

The  second  type  of  data  obtained  from  CHALLENGE  pertained  to  the  content 
of  the  structures.  Examination  of  student  responses  revealed  four  doctrinal ly 
recognized  schemata  used  to  generate  and  organize  items  in  CHALLENGE.  Of 
these  four,  the  most  frequently  used  approach  was  to  organize  the  major  topics 
around  the  factors  used  to  analyze  tactical  missions:  mission,  enemy,  terrain 
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(and  weather),  troops,  and  time  available;  or  METT-T.  Another  popular 
approach  was  to  generate  major  topics  using  the  five  paragraph  headings  in  an 
operations  order  (OPORD):  situation,  mission,  execution,  service  support,  and 
command  and  signal.  A  third  approach  to  organizing  responses  was  to  use  all 
or  a  subset  of  the  eight  troop  leading  procedures  used  to  prepare  a  platoon 
for  an  operation:  receive  and  analyze  the  mission,  issue  the  warning  order, 
form  a  tentative  plan,  start  necessary  movement,  conduct  reconnaissance,  make 
decisions/complete  the  plan,  issue  the  OPORD,  and  supervise  and  refine.  A 
final  scheme  was  to  use  the  acronym  OCOKA  (observation,  cover  and  concealment, 
obstacles,  key  terrain,  and  avenue  of  approach)  to  generate  supporting  points 
usually  under  a  major  topic  relating  to  terrain.  The  output  from  CHALLENGE 
was  examined  to  identify  instances  where  students  used  any  one  of  these 
schemata  to  generate  major  topics  or  supporting  points.  Examination  of  the 
these  data  indicated  that  students  sometimes  used  more  than  one  of  these 
schemata.  For  instance,  several  students  used  the  five  elements  of  METT-T  as 
major  topics  and  the  acronym  OCOKA  for  generating  supporting  points  under  the 
concept  "terrain."  This  finding  suggested  that  student  output  could  be  scored 
two  ways.  First,  a  binary  variable  (1  or  0)  was  used  to  describe  whether  or 
not  students  used  any  of  the  aforementioned  schemata  to  organize  their 
CHALLENGE  responses.  Second,  an  interval  measurement  variable  was  used  to 
describe  the  number  of  detectable  schemata  used  by  each  student.  The  latter 
variable  ranged  in  value  from  0  to  4. 

Results  of  the  Analyses 

Effects  on  confidence  in  the  central  topic.  Mean  ratings  of  confidence 
in  the  central  topic  are  presented  in  Table  9  broken  down  by  group  and  test 
administration.  Inspection  of  the  table  reveals  an  increase  in  confidence 
going  from  pre-  to  posttest.  Analyses  of  these  date  (Appendix  Table  E-3) 
indicated  that  this  difference  was  reliable,  F  (1,  49)  =  14.32,  p  <  .001.  The 
other  evident  trend  is  higher  confidence  ratings  in  the  control  than  in  the 
experimental  group.  This  trend  also  proved  statistically  significant, 

F  (1,  49)  =  4.69,  p  <  .05.  The  means  also  suggested  the  presence  of  a  group  X 
trial  interaction  as  evidenced  by  the  larger  gains  in  confidence  in  the 
experimental  compared  to  the  control  group.  However,  this  interaction  did  not 
prove  to  be  significant,  F  <  1.  Thus,  these  data  failed  to  indicate  an 
increase  in  confidence  as  a  function  of  ARTACT  training. 

Effects  on  schemata  usage.  The  data  in  Table  10  include  means  and 
standard  deviations  of  variables  relating  to  (a)  whether  or  not  students  used 
any  of  the  four  identified  schemata  in  generating  or  organizing  their 
responses  to  the  CHALLENGE  program,  and  (b)  the  number  of  instances  of 
schemata  usage.  Both  performance  variables  are  broken  down  by  group  and  test 
administrations.  The  findings  from  the  two  sets  of  results  are  very  similar 
in  two  respects.  First,  there  is  a  marked  increase  in  the  use  of  schemata  on 
the  second  compared  to  the  first  test  administration.  Separate  ANOVAs  of 
these  two  sets  of  data  (Appendix  Table  E-4)  indicate  that  the  trials  effect 
was  reliable  for  both  the  binary  and  interval  measure  variables,  Fs  (1,  50)  = 
17.60  and  11.00,  respectively,  both  ps  <  .01.  Second,  the  experimental  group 
evidenced  larger  increases  than  the  control  group  in  schemata  usage  across  the 
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Table  9 


Means  and  Standard  Deviations  of  Ratings  of  Confidence  in  the 
Central  Topic 


As  Measured  at 

Group 

Pretest 

Posttest 

Experimental 

M 

77.6 

89.2 

SD 

22.5 

9.3 

Control 

M 

86.9 

93.7 

SD 

11.7 

9.9 

Table  10 

Means  and  Standard  Deviations  of  Variables  Relating  to 
Schemata  to  Organize  Responses  in  CHALLENGE 

Use  of  Doctrinal 

Variability  Source  Group 

As  Measured  at 
Pretest  Posttest 

Presence  of  Any  Schemata 

Experimental 

M 

0.36 

0.84 

SD 

0.49 

0.37 

Control 

M 

0.48 

0.67 

SD 

0.51 

0.10 

Number  of  Instances  of 

Schemata  Use 

Experimental 

M 

0.36 

1.40 

SD 

0.49 

1.08 

Control 

M 

0.48 

1.04 

SD 

0.51 

0.94 
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two  test  administrations.  In  spite  of  the  trends,  however,  the  group  X  trial 
interaction  terms  approached  but  did  not  exceed  conventional  significance 
levels  in  either  case,  Fs  (1*  50)  =  3.46  and  2.84,  respectively,  both 
ps  <  .10. 

Correlations  with  other  knowledge  variables.  Appendix  D  presents  the 
correlations  between  the  binary  variable  used  to  index  students'  use  of 
doctrinal  schemata  in  organizing  their  knowledge  during  the  CHALLENGE  sessions 
and  the  other  knowledge  variables.  Use  of  doctrinal  schemata  as  organizers 
during  the  pre-  and  posttest  sessions  was  positively,  but  not  significantly 
correlated  (p  <  10).  Use  of  doctrinal  schemata  during  the  pretest  session  was 
associated  with  lower  error  rates  on  the  tactical  knowledge  test  administered 
during  this  same  session.  However,  this  relationship  was  not  replicated 
during  the  posttest  session. 

Taking  a  somewhat  different  perspective,  the  correlations  in  Table  D-l 
suggest  that  during  the  pretest  session  the  poorer  performers  on  the  tactical 
knowledge  test  tended  to  be  those  who  were  less  likely  to  use  schemata 
organizers  when  working  on  the  CHALLENGE  program.  However,  the  direction  of 
this  relationship  re/erses  when  pretest  knowledge  test  scores  are  correlated 
with  use  of  schemata  organizers  during  the  posttest  session.  That  is,  higher 
error  rates  on  the  pretest  knowledge  test  were  associated  with  higher 
incidences  of  use  of  schemata  organizers  during  the  posttest  CHALLENGE 
session.  In  other  words,  students  who  made  more  errors  in  the  pretest 
tactical  knowledge  test  became  those  students  most  consistent  in  using 
schemata  organizers  to  describe  their  understanding  of  operations  during  the 
posttest  CHALLENGE  session. 

The  lack  of  relationship  during  the  posttest  session  between  students' 
use  of  doctrinal  schemata  and  the  posttest  knowledge  test  is  probably  due  to 
the  large  percent  of  students  in  both  groups  who  used  doctrinal  schemata 
during  this  session.  That  is,  as  shown  in  Table  10,  84%  of  the  experimental s 
and  67%  of  the  controls  used  one  or  more  schemata  organizers  during  this 
session.  Apparently  this  more  uniform  use  of  schemata  organizers  during  the 
posttest  session  wiped  out  any  relationship  with  posttest  performance  on  the 
tactical  knowledge  test. 

It  should  also  be  noted  that  students  who  used  doctrinal  organizers 
during  the  pretest  session  aid  not  display  an  understanding  that  involved 
interrelating  these  highly  interrelated  organizers.  That  is,  if  one  of  the 
four  doctrinal  schemata  scored  was  used,  it  did  not  incorporate  significant 
parts  of  any  of  the  other  three.  However,  during  the  posttest  session  on 
CHALLENGE,  32%  of  the  experimental  group  and  30%  of  the  control  group 
interrelated  two  to  four  of  the  doctrinal  schemata  in  outlining  their 
understanding  of  planning  a  tactical  operation.  The  difference  between  the 
two  groups  is  not  statistically  significant.  The  importance  of  this  trend  is 
in  documenting  a  change  in  the  ability  to  organize  their  doctrinal  knowledge 
as  a  reflection  of  the  learning  taking  place  and  their  stage  in  this  process. 


26 


Summary 


Results  from  data  analyses  of  the  CHALLENGE  data  may  be  summarized  by 
the  following  generalizations: 

1.  Comparison  of  pre-  and  posttest  performance  reflected  increasing 
expertise  in  tactics  as  indicated  by  (a)  an  increase  in  students'  confidence 
in  their  understanding  of  knowledge  needed  for  planning  tactical  operations, 
and  (b)  increases  in  the  number  of  students  using  doctrinal  schemata  to 
describe  their  understanding  of  planning  tactical  operations  and  in  the  number 
of  such  schemata  used  by  students. 

2.  Despite  some  trends  in  the  hypothesized  direction,  the  analyses  were 
consistent  in  their  failure  to  indicate  any  advantage  of  experimental  over 
control  group,  thereby  failing  to  attribute  changes  in  knowledge  organization 
to  experience  with  ARTACT. 

Effects  on  Performance  in  AOBC 


Data  Scoring  Procedures 

Measures  of  AOBC  classroom  performance  were  divided  into  two  categories. 
The  first  category  comprised  measures  of  overall  performance  in  AOBC  including 
the  students  average  grade  on  all  scored  course  work  and  the  student's 
percentile  rank  within  his  class.  The  second  category  comprised  measures  of 
performance  on  the  Armor  Performance  Test  administered  in  Week  14  of  AOBC, 
subsequent  to  ARTACT  training.  Student  performance  on  this  test  was  recorded 
on  a  pass/fail  basis  for  individual  test  objectives  within  either  the  written 
and  performance  components  of  the  test.  For  the  written  component  of  the 
test,  each  of  the  11  test  objectives  consisted  of  a  single  item.  In  contrast, 
the  11  test  objectives  of  the  performance  components  consisted  of  multiple 
test  items.  Unfortunately,  performance  data  on  individual  items  of  the 
performance  component  were  not  maintained  in  the  student  records. 

Data  on  AOBC  performance  included  the  six  students  (three  experimentals 
and  three  controls)  who  did  not  show  for  the  posttest.  Data  from  these 
students  were  included  because  they  all  had  complete  AOBC  records  and  because 
the  three  students  in  the  experimental  group  had  completed  ARTACT  training. 
However,  the  analyses  did  not  include  the  two  experimental  students  who 
dropped  from  the  program  prior  to  ARTACT  training.  Consequently,  the 
following  comparisons  were  based  on  ns  of  28  and  30  for  experimental  and 
control  groups,  respectively. 

Results  of  the  Analyses 

Effects  on  overall  performance.  Table  11  summarizes  the  overall 
performance  in  AOBC  for  experimental  and  control  groups.  For  both  measures, 
performance  of  the  experimental  group  is  superior  to  that  of  the  control 
aroup.  However,  the  differences  are  not  large,  especially  in  light  of  the 
large  between-subjects  differences,  i.e.,  the  standard  deviations.  Univariate 
analyses  of  these  data  (Appendix  Table  E-5)  indicated  that  in  neither  case 
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Table  11 


Overall  Performance  in  AOBC 


Group _ 

Performance  Measure  Experimental  Control 


Class  Average 

M 

93.0 

91.7 

SD 

3.2 

2.7 

Percentile  Rank 

M 

54.9 

40.8 

SD 

27.8 

26.7 

were  the  differences  between  groups  significant,  although  the  difference  in 
percentile  rank  approached  significance  at  the  .05  level,  F  (1,  56)  -  3.89, 
p  <  .06. 


Effects  on  the  Armor  Performance  Test.  Table  12  displays  the  percent  of 
test  objectives  passed  for  both  the  written  and  performance  components  of  the 
Armor  Performance  Test.  For  both  test  components,  performance  of  the 
experimental  group  was  superior  to  that  of  the  control  group.  Again,  however, 
the  differences  are  not  large  in  reference  to  between- subject  variability. 

The  analysis  on  these  data  (Appendix  Table  E-6)  showed  no  significant 
differences  between  groups  or  between  test  components,  nor  was  the  interaction 
of  these  two  factors  significant. 


Table  12 

Percent  Objectives  Passed  for  the  Two  Components  of  the  Armor 
Performance  Test 


Test  Component 

Group 

Experimental 

Control 

Written 

M 

86.0 

84.2 

SD 

12.0 

9.8 

Performance 

M 

83.7 

77.7 

SD 

17.2 

17.5 
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The  next  analysis  addressed  differences  between  objectives  within  the 
written  component.  Of  the  eleven  objectives  within  this  component,  two  were 
related  to  ARTACT:  one  to  the  Offensive  Operations  module  and  the  other  to 
the  Defensive  Operations  module.6  The  remaining  nine  objectives  were  not 
related  to  ARTACT.  The  top  portion  of  Table  13  indicates  that  there  is  an 
apparent  tendency  for  the  differences  between  groups  to  be  larger  for  the 
objectives  that  are  related  to  ARTACT.  However,  examination  of  performance  on 
individual  objectives  indicates  that  this  overall  tendency  appeared  only  on 
the  item  related  to  Offensive  Operations  and  not  on  the  item  related  to 
Defensive  Operations.  Analyses  of  these  data  (Appendix  Table  E-7)  showed  no 
significant  effects  due  to  group,  the  type  of  test  objective,  or  their 
interaction. 


Table  13 

Performance  on  Objectives  Within  Written  and  Performance  Components  of 
the  Armor  Performance  Test 


Group 

Relation  of  Objective 

To  ARTACT 

Experimental 

Control 

Written  Component 

Related 

M 

85.7 

81.7 

SD 

26.7 

27.8 

Not  Related 

M 

86.1 

84.7 

SD 

12.7 

10.3 

Performance  Component 

Most  Items  Related 

M 

82.1 

70.0 

SD 

24.4 

31.1 

Half  of  Items  Related 

M 

78.6 

71.7 

SD 

31.7 

36.4 

Few  of  Items  Related 

M 

80.9 

76.6 

SD 

24.7 

26.5 

None  of  Items  Related 

M 

87.5 

86.7 

SD 

32.3 

34.6 

*For  reasons  of  test  security,  the  specific  items  cannot  be  revealed. 
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Differences  between  objectives  in  the  performance  component  were  more 
difficult  to  describe  due  to  the  fact  that  objectives  comprised  multiple 
items.  Some  of  these  items  were  related  to  ARTACT  skills  and  knowledges,  and 
some  were  not.  A  simple  scheme  was  devised  to  describe  this  relationship  by 
creating  four  categories  of  objectives  defined  as  having  most,  half,  few,  or 
none  of  the  items  related  to  ARTACT.  As  a  result  of  this  exhaustive 
classification  scheme,  at  least  two  objectives  were  assigned  to  each  of  these 
four  categories  with  the  few  category  consisting  of  three  objectives.  The 
bottom  portion  of  Table  lTl'ndicates  a  direct  relationship  between  the 
experimental /control  differences  and  the  degree  to  which  the  objectives  were 
related  to  ARTACT.  That  is,  the  experimental  group's  advantage  was  larger  for 
test  objectives  that  were  more  closely  related  to  ARTACT.  Despite  these 
promising  trends  evident  in  the  means,  none  of  the  effects  of  the  analyses 
were  significant  (Appendix  Table  E-7).  The  nonsignificance  of  these  data  may 
be  attributed  to  the  large  between-student  variability  as  evidenced  by  the 
standard  deviation  values.  The  large  variability,  in  turn,  is  partly  due  to 
the  fact  that  performance  within  each  category  was  based  on  only  two  or  three 
test  items. 

Correlations  with  other  knowledge  measures.  According  to  conventional 
standards  for  statistical  significance,  the  only  reliable  relationship  between 
the  Armor  Performance  Test  and  other  knowledge  measures  was  the  negative 
correlation  between  performance  on  the  written  component  of  the  Armor 
Performance  Test  and  errors  on  the  tactical  knowledge  pretest,  r  -  -.30, 
p  <  .04.  In  other  words,  higher  scores  on  the  written  component  were 
associated  with  lower  error  rates  on  the  knowledge  pretest.  A  similar 
negative  relationship  was  evident  between  performance  on  the  written  component 
and  performance  on  the  tactical  knowledge  posttest,  but  it  was  not  signifi¬ 
cant,  p  <  .08.  Other  correlations  with  the  Armor  Performance  Test  that 
approached  but  did  not  exceed  significance  at  the  .05  level  included  (a)  the 
positive  correlation  between  the  two  components  of  the  Armor  Performance  Test, 
p  <  .06;  (b)  the  positive  correlation  between  the  written  component  and 
specific  knowledge  comments,  p  <  .07,  and  (c)  the  positive  correlation  between 
the  performance  component  and  specific  knowledge  comments,  p  <  .10.  Thus,  the 
overall  pattern  of  intercorrelations  support  the  validity  of  these  measures, 
including  the  Armor  Performance  Test,  as  indexes  of  individual  differences  in 
tactical  knowledge. 

Summary 

The  trends  evident  in  the  descriptive  statistics  suggested  that 
experimental  students  performed  better  than  control  students  on  all  measures 
of  A0BC  performance.  Furthermore,  the  performance  component  of  the  Armor 
Performance  Test  indicated  larger  experimental-control  differences  for  those 
test  objectives  that  were  most  related  to  ARTACT.  Despite  the  consistency  of 
these  trends,  none  were  statistically  significant.  Thus,  the  A0BC  data  failed 
to  unambiguously  show  that  experience  with  ARTACT  positively  transfers  to  A0BC 
performance  and  written  tests. 
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Formative  Evaluation: 

1.  Performance  of  ARTACT  Hardware  and  Software 


Results 


The  most  frequently  occurring  hardware/software  problem  encountered 
during  the  evaluation  was  a  module's  "freezing,"  i.e.,  stopping  at  some  point 
in  program  and  not  responding  to  student  input.  These  freezes  usually 
required  the  experimenter  to  restart  the  program  from  the  beginning.  Table  14 
and  the  following  paragraphs  elaborate  on  problems  experienced  in  each  of  the 
four  ARTACT  programs: 

1.  Planning  FA  Fires.  No  problems  were  experienced  when  administering 
the  Planning  FA  Fires  module. 

2.  Techniques  of  Movement.  The  Techniques  of  Movement  program  froze 
while  being  administered  to  six  of  the  seventeen  students  in  the  first  three 
classes.  The  cause  of  the  problem  was  eventually  corrected,  and  the  program 
did  not  freeze  at  all  when  administered  to  the  students  in  the  last  three 
classes.  The  program  was  restarted  whenever  it  froze,  and  all  students  were 
able  to  complete  the  module. 

3.  Offensive  Operations.  The  Offensive  Operations  module  did  not  run 
properly  when  it  first  arrived,  and  the  problem  was  not  corrected  in  time  to 
administer  the  program  to  the  students  in  the  first  class.  Consequently,  the 
class  was  dropped  from  the  study.  The  program  was  sufficiently  improved  so 
that  it  could  be  administered  to  the  students  in  the  second  class,  but  the 
problems  were  never  totally  corrected.  The  program  froze  at  the  very  end  for 
all  six  students  in  the  second  class  and  for  three  of  the  five  students  in  the 
third  class.  Although  these  nine  students  completed  the  program  before  it 
froze,  they  were  unable  to  receive  a  printout  of  their  performance,  and  all 
records  of  their  performance  were  lost.  The  program  also  froze  four  times 
prior  to  the  end  while  it  was  being  administered  to  students  in  the  third, 
fifth,  and  sixth  classes,  and  it  had  to  be  restarted  each  time.  Since  the 
program  ran  properly  after  being  restarted,  printouts  were  available  for  all 
four  students,  and  all  data  were  saved. 

4.  Defensive  Operations.  Although  the  Defensive  Operations  module  did 
not  run  when  it  first  arrived,  the  program  was  sufficiently  improved  so  that 
it  could  be  used.  However,  the  program  froze  four  times  prior  to  the  end.  It 
was  restarted  each  time  and  then  ran  properly.  Consequently,  printouts  were 
available  for  all  students,  and  all  data  were  saved. 
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Table  14 

Number  of  Students  Experiencing  a  Program  Freeze  During  the  Implementation 
of  an  ARTACT  Program 

AOBC  Class 

ARTACT  Program  I  l  3  4  5  Z  Total 


Planning  FA  Fires 

Froze  During  Program 
Froze  at  End  of  Program 
Did  Not  Freeze 

Total  Number  of  Students 
Techniques  of  Movement 

Froze  During  Program 
Froze  at  End  of  Program 
Did  Not  Freeze 
Total  Number  of  Students 
Offensive  Operations 

Froze  During  Program 
Froze  at  End  of  Program 
Did  Not  Freeze 
Total  Number  of  Students 
Defensive  Operations 

Froze  During  Program 
Froze  at  End  of  Program 
Did  Not  Freeze 
Total  Number  of  Students 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

_6_ 

6 

5 

5 

6 

6 

34 

6 

6 

5 

5 

6 

6 

34 

1 

2 

3 

0 

0 

0 

6 

0 

0 

0 

0 

0 

0 

0 

5 

4 

2 

_5_ 

6 

6 

28 

6 

6 

5 

5 

6 

6 

34 

0 

1 

0 

1 

2 

4 

6 

3 

0 

0 

0 

9 

_0_ 

1 

5 

5 

4 

15 

6 

5 

5 

6 

6 

28 

0 

1 

1 

1 

1 

4 

0 

0 

0 

0 

0 

0 

5 

4 

4 

5 

5 

23 

5 

5 

5 

6 

6 

27 
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Summary 


The  Planning  FA  Fires  program  was  free  of  problems  throughout  the  field 
test.  Initial  problems  that  caused  the  Techniques  of  Movement  program  to 
freeze  were  eliminated  early  in  the  field  test.  However,  problems  with  the 
Offensive  Operations  and  Defensive  Operations  programs  continued  sporadically 
throughout  the  field  test.  The  influence  these  problems  may  have  had  on 
students'  subjective  reactions  to  ARTACT  will  be  considered  in  the  next 
section. 


Formative  Evaluation: 

2.  Responses  to  Structured  Questionnaire  Items 


Appendix  A-2  presents  the  version  of  the  questionnaire  given  to  the 
experimental  students  during  the  posttest  session.  The  version  given  to  the 
control  students  contained  only  the  first  five  items  which  deal  with  AOBC  but 
not  ARTACT.  These  five  items  were  not  included  in  the  questionnaire 
administered  to  the  students  from  the  second  AOBC  class,  i.e.,  the  first  class 
whose  data  were  included  in  the  analysis.  Consequently,  data  on  these  items 
were  obtained  from  19  students  in  the  experimental  group  and  21  in  the  control 
group.  The  questionnaire  contained  three  types  of  items  requiring  different 
scoring  procedures,  each  of  which  is  described  below. 

The  first  item  in  the  questionnaire  required  the  students  to  rate  the 
difficulty  they  experienced  in  understanding  each  of  11  topics  of  instruction 
in  AOBC.  Students  were  instructed  to  make  their  ratings  on  a  100-point  scale 
where  "1"  was  extremely  easy  and  "100"  was  extremely  difficult.  Examination 
of  the  training  objectives  associated  with  the  11  AOBC  topics  revealed  that 
they  could  be  classified  as  either  related  or  unrelated  to  ARTACT' s  content. 
Related  topics  were  those  six  AOBC  content  areas  (Table  15)  that  were  covered, 
at  least  in  part,  by  one  or  more  ARTACT  modules.  Unrelated  topics  were  those 
remaining  five  AOBC  content  areas  (Table  15)  that  were  not  covered  by  ARTACT. 

The  next  group  of  structured  items  on  the  questionnaire  consisted  of 
(a)  four  questions  designed  to  assess  the  subjective  reactions  of  the 
experimental  and  control  groups  toward  AOBC,  and  (b)  five  questions  designed 
to  assess  the  reactions  of  the  experimental  group  toward  ARTACT.  Students 
responded  to  each  of  these  items  using  a  5-point  scale.  The  responses  were 
coded  surh  that  "5"  represented  to  most  positive  response  and  "1,"  the  least 
positive. 

The  final  two  structured  items  on  the  questionnaire  were  summary 
questions  asking  students  in  the  experimental  aroup  (a)  to  specify  which 
ARTACT  program  they  found  most  helpful,  and  (b)  to  state  whether  or  not  they 
believed  ARTACT  should  be  adopted  in  AOBC.  These  two  forced-choice  items  were 
scored  by  counting  the  number  of  students  selecting  each  alternative. 

Results  of  the  Analyses 

Effects  on  rated  difficulty  of  AOBC  topics.  The  ARTACT  programs  were 
designed  to  be  administered  at  various  points  in  training  prior  to  receiving 
the  regular,  related  armor  tactical  training.  Assuming  the  ARTACT  programs 
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were  effective  in  enabling  students  to  more  readily  comprehend  the  regular 
instruction  in  AOBC,  it  was  expected  that  experimental  students  would  rate  the 
related  topics  easier  to  understand  than  would  the  control  students. 

Table  15  presents  means  and  standard  deviations  of  ratings  on  each  topic 
broken  down  by  experimental  and  control  conditions.  These  data  indicated  that 
the  groups  rated  related  and  unrelated  topics  in  a  manner  that  was  contrary  to 
expectations:  Whereas  the  experimental  and  control  group  did  not  differ  very 
much  in  their  ratings  of  unrelated  topics  (means  =  37.8  and  38.7,  respec¬ 
tively),  the  experimental  group  rated  the  related  topics  substantially  more 
difficult  than  control  group  (means  =  44.9  and  34.6,  respectively).  The  data 
were  subjected  to  analysis  with  students  nested  into  either  experimental  or 
control  groups  and  crossed  with  the  11  question  items  (i.e.,  AOBC  topics). 

The  results  of  the  analysis  are  contained  in  Appendix  Table  E-8. 

A  result  from  this  analysis  that  is  relevant  to  the  present  hypothesis 
was  the  significant  interaction  between  topic  relatedness  (related,  unrelated) 
and  groups,  F  (1,  38)  =  4.10,  p  <  .05.  Analysis  of  individual  means  indicated 
that  two  differences  approached  but  did  not  exceed  conventional  values  for 
statistical  significance:  (a)  the  difference  between  experimental  and  control 
groups  in  their  ratings  of  related  topics,  t  (38)  =  1.36,  .20  >  p  >  .10;  and 
(b)  the  difference  between  related  and  unrelated  topics  for  the  experimental 
group,  t  (18)  «  1.83,  p  <  .10.  Despite  the  failure  to  obtain  reliable 
differences  between  individual  means,  the  significant  interaction  between 
topic  relatedness  and  groups  confirmed  that  the  differences  between  the 
ratings  of  related  and  unrelated  topics  were  dependent  upon  group  assignment. 
Thus,  the  ratings  made  by  students  who  had  received  ARTACT  training  prior  to 
the  related  classroom  instruction  suggest  they  perceived  the  AOBC  instruction 
as  more  difficult  to  understand  than  did  the  students  who  did  not  receive 
ARTACT  training.  Speculations  on  what  this  may  signify  will  be  addressed  in 
the  Discussion  section  of  this  report. 

Effects  on  subjective  reactions  to  AOBC.  Means  and  standard  deviations 
of  subjective  reactions  to  AOBC  are  displayed  in  the  upper  half  of  Table  16. 
The  means  indicate  that  students  reacted  generally  favorably  toward  AOBC  with 
no  apparent  differences  in  reaction  to  individual  questions.  Analyses  of  the 
data  (Appendix  Table  E-9)  indicated  no  reliable  differences  in  ratings  due  to 
group  or  to  question  item,  and  no  differences  due  to  the  group  X  question 
interaction. 

Subjective  reactions  to  ARTACT.  Means  and  standard  deviations  of  the 
subjective  reactions  to  ARTACT  are  displayed  in  the  bottom  half  of  Table  16. 
Although  the  means  indicate  that  students  reacted  generally  favorably  toward 
ARTACT,  there  are  more  marked  differences  in  student  reactions  to  questions. 
Post  hoc  comparisons7  indicated  that  students  rated  the  useability  of  the 
computer  more  highly  than  the  intrinsic  interest,  helpfulness,  or  effective¬ 
ness  of  the  ARTACT  programs,  t  (24)  =  3.24,  p  <  .004.  In  contrast, 


7Using  the  Bonferroni  procedure,  alpha  was  adjusted  by  dividing  by  the 
number  of  possible  comparisons  (six).  The  resulting  significance  criterion 
for  these  comparisons  was  .0083. 
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Table  15 


Means  and  Standard  Deviations  of  Difficulty  Ratings  of  Topics  in  AOBC* 

Group 

Topic  of  Instruction 

Experimental 

Control 

Related  to  ARTACT 

Command  and  Control 

M 

34 

33 

SD 

23.8 

27.2 

Techniques  of  Movement 

M 

37 

28 

SD 

30.0 

29.0 

Cavalry  Operations 

M 

53 

45 

SD 

28.8 

28.7 

Fundamentals  of  Reconnaissance 

M 

45 

36 

SD 

27.8 

25.3 

Fundamentals  of  Offense 

M 

51 

32 

SD 

29.4 

29.7 

Fundamentals  of  Defense 

M 

50 

33 

SD 

29.6 

29.7 

Unrelated  to  ARTACT 

Engineers  and  Mobil ity/Counter-Mobility 

Operations 

M 

29 

42 

SD 

24.7 

30.8 

Call  For  and  Adjust  Indirect  Fire 

M 

61 

61 

SD 

29.2 

29.7 

Tactical  Air  Operations 

M 

31 

31 

SD 

32.8 

27.0 

NBC  Defensive  Operations 

M 

32 

25 

SD 

27.2 

22.2 

Fundamentals  of  Security 

M 

36 

34 

SD 

25.7 

23.3 

'Ratings  were  made  on  a  scale  from  1  to  100  with  higher  ratings  indicating 
greater  difficulty. 
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Table  16 


Means  and  Standard  Deviations  of  Responses  to  Questionnaire  Items  Pertaining 
to  AOBC  and  ARTACT  Rated  on  a  Five-Point  Scale* 


*A  rating  of  "5"  indicated  most  favorable  reaction  and  "1",  least  favorable. 

bThe  ns  for  the  items  pertaining  to  AOBC  were  19  for  the  experimental  group 
and  21  for  the  control  group. 

cThe  n  for  the  items  pertaining  to  ARTACT  was  25. 
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differences  in  ratings  between  the  latter  three  questions  were  not 
significant. 

Effects  of  software/hardware  problems  on  subjective  reactions  to  ARTACT. 
Add i tional  analyses  were  performed  to  determine  if  student  reactions  to  ARTACT 
were  negatively  influenced  by  the  software  and  hardware  problems  that  occurred 
during  ARTACT  training.  The  students  in  the  experimental  group  were  separated 
into  two  groups  based  on  the  need  to  restart  a  program  during  ARTACT  training. 
None  of  the  modules  had  to  be  restarted  for  fifteen  students  of  the  students, 
while  at  least  one  module  had  to  be  restarted  for  the  other  ten  students. 

Means  and  standard  deviations  of  reactions  to  ARTACT  are  displayed  in  Table  17 
for  each  of  the  two  groups  of  students.  The  means  indicate  that  there  are 
marked  differences  in  the  reactions  of  the  two  groups  on  all  four  items  with 
the  students  in  the  restart  group  showing  the  more  favorable  reaction  to 
ARTACT.  This  finding  was  certainly  contrary  to  expectations.  A  post  hoc 
analysis  of  these  data  (Appendix  Table  E-10)  indicate  significant  main  effects 
for  group  (f  (1,  43)  =  4.43,  p  <  .05)  and  for  questions  [F  (3,69)  =  8.07, 
p  <  .001),  but  not  a  significant  interaction  between  these  two  factors. 


Table  17 

Means  and  Standard  Deviations  of  Responses  to  Questionnaire  Items  Concerning 
ARTACT  for  Students  Who  Restarted  an  ARTACT  Program  and  Those  Who  Did  Not 


Restarted  an  ARTACT  Program* 

Questionnaire  Item  Yes  No 


How  helpful  were  the  ARTACT  programs  as  a 


way  of  learning  about  platoon  tactics? 

M 

3.9 

3.1 

SD 

1.22 

1.36 

How  hard  was  it  to  learn  to  use  the  computer 

to  work  on  the  ARTACT  programs? 

M 

4.9 

3.9 

SD 

0.30 

1.18 

How  interesting  were  the  ARTACT  programs? 

M 

3.6 

3.0 

SD 

1.11 

1.26 

How  effective  were  the  ARTACT  programs  in 
getting  you  to  think  about  platoon  tactics? 

M 

4.10 

3.2 

SD 

0.83 

1.22 

*The  ns  were  10  for  the  group  that  restarted  an  ARTACT  program  and  15  for  the 
group  that  did  not. 
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Responses  to  ARTACT  summary  questions.  Table  18  presents  the  student 
responses  to  the  two  summary  questions  contained  in  the  questionnaire.  The 
responses  to  the  question  concerning  the  most  helpful  ARTACT  program  revealed 
that  the  majority  of  the  students  found  Planning  FA  Fires  to  be  most  helpful. 
This  finding  was  unexpected  given  that  module  has  the  least  overlap  with  AOBC 
instruction.  Perhaps  students  found  ARTACT  most  helpful  when  it  provided  new 
information.  The  responses  also  revealed  that  none  of  the  students  chose 
Defensive  Operations  as  the  most  helpful  program.  The  Defensive  Operations 
program  was  the  last  one  to  be  presented  to  the  students.  Consequently,  they 
already  had  been  exposed  to  the  other  three  programs,  and  they  had  almost 
completed  their  classroom  training  when  exposed  to  the  Defensive  Operations 
program.  Given  these  circumstances,  it  is  possible  that  the  students 
perceived  the  program  to  be  the  least  helpful  because  they  had  already  learned 
much  of  the  information  it  contained. 


Table  18 

Responses  of  Students  in  Experimental  Group  to  Summary  Questions 
Concerning  ARTACT 


Question  Items 

Alternative  Responses 

f 

% 

Which  ARTACT  program  did  you  find  most  helpful? 

Planning  FA  Fires 

13 

59 

Offensive  Operations 

5 

23 

Techniques  of  Movement 

4 

18 

Defensive  Operations 

0 

22* 

0 

In  your  opinion,  should  the  ARTACT  programs 
become  part  of  the  AOBC  curriculum? 

Yes 

6 

24 

Yes,  but  only  if  they  are  improved 

13 

52 

No 

6 

25 

24 

"Although  25  students  responded  to  this  item,  the  responses  made  by  three 
students  were  uninterpretable. 
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The  second  summary  question  involved  their  opinion  concerning  the 
implementation  of  the  ARTACT  programs  in  AOBC.  The  majority  of  students  (76%) 
stated  that  ARTACT  should  be  implemented,  although  52%  felt  that  it  should  be 
implemented  only  if  the  programs  are  improved.  To  determine  if  the  software 
and  hardware  problems  affected  their  opinions  concerning  the  implementation  of 
the  program,  the  students  were  separated  into  two  groups  based  on  the  need  to 
restart  a  program  during  ARTACT  training.  Table  19  presents  the  student 
responses  to  this  question  for  the  two  groups  of  students.  The  data  appear  to 
show  that  fewer  students  rejected  the  implementation  of  ARTACT  in  AOBC  if  they 
had  to  restart  the  program  during  training.  Due  to  the  small  frequencies  in 
most  of  the  cells,  the  number  of  students  who  responded  "YES"  and  the  number 
who  responded  "YES,  BUT  ONLY  IF  THEY  ARE  IMPROVED"  were  summed,  and  a  Fisher's 
exact  test  was  performed  to  compare  the  response  frequencies  of  the  two 
groups.  The  result  showed  that  the  difference  in  the  responses  between  the 
two  groups  was  not  significant  ( p  =  .35). 


Table  19 

Responses  to  Summary  Question  Concerning  Implementation  of  ARTACT  in  AOBC 
for  Students  Who  Restarted  and  Did  Not  Restart  an  ARTACT  Program 


Restarted  an 

ARTACT  Program 

Questionnaire  Item 

Yes 

No 

f 

% 

f 

% 

In  your  opinion,  should  the  ARTACT 
programs  become  part  of  the  AOBC 
curriculum? 

Yes 

4 

40 

3 

20 

Yes,  but  only  if  they  are  improved 

5 

50 

7 

47 

No 

1 

10 

5 

33 

10 

15 

Summary 

Results  from  the  analyses  of  responses  to  structured  questionnaire  items 
may  be  summarized  as  follows: 

1.  The  experimental  and  control  groups  differed  in  their  ratings  of  the 
difficulty  of  AOBC  topics.  Contrary  to  expectations,  the  experimental  group 
rated  topics  that  were  related  to  prior  tutoring  received  in  ARTACT  as  more, 
rather  than  less,  difficult  than  did  the  control  group.  The  groups  did  not 
differ  with  respect  to  their  ratings  of  topics  not  related  to  tutoring 
received  in  ARTACT. 
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2.  Experimental  and  control  group  were  equally  favorable  in  their 
ratings  of  the  interest  level  of  classroom  instruction  ("quite  interesting") 
and  the  extent  to  which  they  were  prepared  for  the  10-day  war,  for  planning 
platoon  tactics,  and  for  serving  in  combat  as  a  tank  platoon  leader  ("quite 
prepared"). 


3.  Average  ratings  of  ARTACT  programs  made  by  students  in  the 
experimental  group  showed  that  the  programs  were  perceived  as  "somewhat" 
interesting  and  helpful  in  learning  about  platoon  tactics.  They  were  rated 
slightly  higher  ("somewhat  to  quite"  effective)  in  their  effectiveness  in 
getting  them  to  think  about  platoon  tactics.  Learning  how  to  use  the  computer 
to  run  ARTACT  was  not  perceived  to  be  difficult.  Surprisingly,  the  ARTACT 
students  rated  the  Planning  FA  Fires  module  as  the  most  helpful  of  the  four 
modules  even  though  the  topic  was  no  longer  presented  in  AOBC. 

4.  The  majority  (76%)  of  the  students  in  the  experimental  group  felt 
that  ARTACT  should  become  part  of  the  AOBC  curriculum,  although  two-thirds  of 
these  students  were  in  favor  of  its  adoption  only  if  the  programs  are 
improved.  Opinions  on  the  types  of  improvements  considered  important  are 
discussed  in  the  section  following  this  summary. 

5.  The  ratings  of  the  ARTACT  programs  were  subjected  to  post-hoc 
analyses  to  determine  if  students  who  experienced  software  and  hardware 
problems  had  less  favorable  reactions  to  ARTACT  than  students  who  did  not 
experience  these  problems.  Contrary  to  expectations,  the  results  showed  that 
students  who  experienced  such  problems  had  more  favorable,  not  less  favorable, 
reactions  to  ARTACT.  However,  the  presence  or  absence  of  these  problems  did 
not  relate  to  student  opinions  concerning  the  adoption  of  ARTACT  in  AOBC. 

Formative  Evaluation: 

3.  Responses  to  Unstructured  Questionnaire  Items 
Data  Scoring  Procedures 

Responses  of  students  in  the  experimental  group  to  the  four  open-ended 
questionnaire  items  were  read  through  for  content,  and  tentative  response 
categories  were  proposed.  Student  responses  were  then  sorted  into  the 
categories.  The  categories  were  examined  for  purity  and  for  the  accuracy  of 
category  labels.  Appropriate  changes  to  the  categories  were  made,  and  the 
items  were  resorted  into  the  final  configuration.  To  examine  some  of  the 
factors  that  perhaps  related  to  student's  reactions  to  these  questions, 
students  in  the  experimental  group  were  classified  three  ways:  (a)  whether  or 
not  they  experienced  problems  during  ARTACT,  (b)  whether  they  were  in  the  top 
or  bottom  half  of  their  AOBC  classes,  and  (c)  whether  students  were  above  or 
below  the  median  in  improvement  from  the  first  to  the  second  administration  of 
the  tactical  knowledge  test. 
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Results  of  the  Analyses 


Three  of  the  four  open-ended  questionnaire  items  asked  the  students  in 
the  experimental  group  to  describe  what  they  liked  most  about  the  ARTACT 
programs,  what  they  disliked  most  about  the  programs,  and  what  improvements  or 
changes  they  felt  should  be  made.  The  final  unstructured  item  provided  the 
students  an  opportunity  to  make  any  other  comments  or  suggestions.  The 
responses  to  these  four  items  are  summarized  in  Table  20  and  are  described 
briefly  below.  A  more  detailed  analysis  of  student  responses  to  the  four 
items  and  breakdowns  according  to  uncontrolled  events  and  student 
characteristics  are  provided  in  Appendix  6. 

Most  liked  aspects  of  ARTACT.  The  positive  aspects  of  ARTACT  that  were 
cited  most  often  were  its  role  in  helping  to  prepare  for  AOBC  classes  (nine 
students)  and  the  feedback  that  the  programs  provided  (six  students).  Another 
category  of  answers  focused  on  specific  characteristics  of  the  programs  that 
some  students  particularly  liked  such  as  the  graphics  or  the  way  the  content 
was  organized  (five  students).  Responses  to  this  item  (Appendix  Table  G-l) 
were  unrelated  to  the  students'  rank  in  their  AOBC  class  except  for  one 
response  category  where  the  relationship  approached  statistical  significance: 
Seven  of  the  thirteen  students  in  the  top  half  of  their  class  praised  ARTACT' s 
role  in  helping  to  prepare  for  AOBC  compared  to  just  two  of  the  twelve 
students  in  the  bottom  half  of  their  class,  p  =  .097.  This  finding  suggests 
that  the  better  students  tended  to  recognize  the  important  role  that  ARTACT 
could  serve  in  AOBC.  Finally,  responses  to  this  item  were  unrelated  to  the 
experience  of  a  software  or  hardware  problem  on  ARTACT,  i . e . ,  having  to 
restart  a  program. 

Most  disliked  aspects  of  ARTACT.  The  most  frequent  complaint  about 
ARTACT  dealt  with  problems  in  scheduling  and  the  time  required  (six  students). 
Other  complaints  focused  on  deviations  from  doctrine  (eight  students), 
specific  aspects  of  the  programs  that  some  students  particularly  disliked  such 
as  the  emphasis  on  movement  formations  and  techniques  (six  students),  and 
ARTACT' s  rigidity  in  accepting  student  responses  (seven  students).  Responses 
to  this  item  (Appendix  Table  G-2)  were  unrelated  to  either  the  students'  rank 
in  their  AOBC  class  or  the  experience  of  a  software  or  hardware  program  on 
ARTACT. 

Recommended  improvements  and/or  changes  in  ARTACT  programs.  The  most 
frequent  answers  dealt  with  improving  various  program  characteristics  such  as 
increasing  their  flexibility  in  accepting  answers  and  in  allowing  deviations 
from  doctrine  (six  students).  Other  recommendations  that  were  frequently  made 
focused  on  improving  or  changing  the  software  or  hardware  (six  students),  the 
map  display  (four  students),  the  feedback  (four  students),  and  the  scenarios 
(four  students).  Responses  to  this  item  (Appendix  Table  G-3)  were  unrelated 
to  the  students'  rank  in  their  AOBC  class  except  for  one  response  category: 
Four  students  in  the  top  half  of  their  class  recommended  improvements  or 
changes  in  the  feedback  provided  by  ARTACT  compared  to  none  in  the  bottom  half 
of  their  class,  p  =  .039.  The  relationship  approached  statistical 
significance  on  one  other  response  category:  Only  one  student  in  the  top  half 
of  his  AOBC  class  recommended  software  or  hardware  improvements  compared  to 
five  students  in  the  bottom  half  of  their  class,  p  =  .073.  These  findings 


41 


Table  20 

Responses  to  Unstructured  Questionnaire  Items  Presented  by  Response  Cluster 


Questionnaire  Item 

Response  Cluster 

Number  of 

Students 

Responding 

Number  of 
Responses 

Made 

What  do  you  like  most  about  the  ARTACT  programs? 

Preparation  for  the  AOBC  Classes 

9 

10 

Program  Characteristics 

5 

8 

Feedback 

6 

7 

Information  Learned 

4 

4 

Research  Personnel 

3 

3 

Miscellaneous 

3 

3 

No  Response 

2 

What  do  you  dislike  most  about  the  ARTACT 
programs? 

Scheduling 

6 

10 

Differences  from  Doctrine 

8 

9 

Specific  Aspects  of  the  Programs 

6 

9 

Rigidity  of  Programs 

7 

8 

Quality  of  Map  Display 

4 

4 

Equipment  Problems 

3 

3 

Miscellaneous 

3 

3 

No  Response 

2 

What  Improvements  or  changes  should  be  made 
in  the  ARTACT  programs? 

Program  Characteristics 

6 

10 

Software/Hardware 

6 

7 

Map  Display 

4 

5 

Feedback 

4 

5 

Scenarios  and  Missions 

4 

5 

Uses  for  ARTACT 

3 

5 

Consistency  with  Classroom  and  Doctrine 

4 

4 

Scheduling 

3 

3 

Miscel laneous 

2 

2 

No  Response 

3 

Other  comments  or  suggestion 

Negative  Comments 

5 

13 

Suggestions 

7 

8 

Comments  on  ARTACT  Staffmembers 

4 

6 

Positive  Comments 

4 

5 

No  Response 

11 
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suggest  that  the  better  AOBC  students  tended  to  focus  on  the  benefits  that 
ARTACT  could  provide  in  AOBC,  whereas  the  poorer  students  tended  to  focus  on 
the  reliability  of  the  system.  Finally,  responses  to  this  item  were  unrelated 
to  the  experience  of  a  software  or  hardware  problem  on  ARTACT. 

Other  comments  and  suggestions.  Although  all  but  two  or  three  students 
responded  to  the  questions  summarized  above,  only  14  of  the  25  students 
responded  to  this  final  question.  One  response  category  contained  specific 
suggestions  such  as  the  suggestion  that  ARTACT  training  be  scheduled  on 
weekday  evenings  or  that  students  should  be  able  to  repeat  a  part  of  the 
program  after  their  mistakes  have  been  identified  (seven  students).  The  other 
categories  contained  negative  comments  relating  to  ARTACT  (five  students), 
positive  comments  relating  to  ARTACT  (four  students),  and  positive  comments 
relating  to  the  evaluation  staff  (four  students).  Responses  to  this  item 
(Appendix  Table  G-4)  were  unrelated  to  the  students'  rank  in  their  AOBC  class. 
They  were  also  unrelated  to  the  experience  of  a  software  or  hardware  problem 
on  ARTACT  except  for  one  response  category:  Four  of  the  ten  students  who 
restarted  a  program  praised  the  evaluation  staff  compared  to  none  of  the 
fifteen  students  who  did  not  restart  a  program,  p  =  .017.  This  finding 
suggests  that  the  students  appreciated  the  efforts  of  the  research  staff  to 
provide  each  student  an  opportunity  to  complete  all  four  ARTACT  modules. 

Summary 

The  responses  made  to  the  four  open-ended  questions  can  be  summarized  as 
follows: 


1.  The  students  in  the  experimental  group  had  both  positive  and 
negative  reactions  to  ARTACT. 

2.  While  many  of  ARTACT 's  characteristics  were  praised,  no  single 
characteristic  was  cited  frequently.  Many  of  the  students  reacted  favorably 
toward  the  learning  experiences  that  ARTACT  provided,  particularly  toward  its 
role  in  helping  them  prepare  for  AOBC. 

3.  Many  of  the  students  saw  the  need  to  improve  the  programs. 

Deviations  from  doctrine  and  the  rigidity  of  the  programs  were  common 
complaints.  Although  scheduling  was  a  frequently  mentioned  problem,  it  would 
probably  not  remain  one  if  ARTACT  were  adopted  by  AOBC. 

4.  Students  who  were  most  successful  in  AOBC  (i.e.,  ranked  in  the  top 
half  of  their  class)  recommended  improvements  in  the  feedback  provided  by 
ARTACT  and  provided  more  praise  for  ARTACT 's  role  in  helping  them  prepare  for 
AOBC  classes.  Students  who  were  least  successful  in  AOBC  (i.e.,  ranked  in  the 
bottom  half  of  their  class)  recommended  improvements  in  ARTACT 's  software  and 
hardware. 
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General  Discussion 


The  procedure  used  to  experimentally  implement  ARTACT  merits  further 
discussion.  In  order  to  experimentally  control  exposure  to  ARTACT,  students 
in  the  experimental  group  were  required  to  attend  the  training  sessions  for 
all  four  modules,  each  under  the  supervision  of  the  research  staff.  Students 
were  not  allowed  any  additional  voluntary  exposures  to  ARTACT.  In  contrast  to 
this  procedure,  ARTACT  was  designed  to  be  used  by  the  students  alone  and  on  an 
ad  libitum  basis.  One  could  argue  that  the  procedure  used  in  the  present 
study  caused  ARTACT  to  be  implemented  suboptimally  because  it  allowed  no  more 
that  6-8  hours  of  contact  with  the  system.  On  the  other  hand,  AOBC  students 
are  relatively  busy  during  the  16  weeks  of  their  training.  Given  these  time 
pressures,  it  is  likely  that  many  students  would  not  have  completed  all  four 
modules  voluntarily.  In  that  regard,  the  experimental  procedure  insured  that 
all  students  in  the  experimental  group  had,  at  least,  the  minimum  exposure  to 
ARTACT;  i.e.,  they  completed  each  of  the  four  modules.  In  either  case, 
interpretation  of  the  effects  of  ARTACT  should  be  tempered  by  the  fact  that  it 
was  not  implemented  in  the  manner  that  it  was  designed  to  be  used  in  practice. 

Effects  on  Retention  of  Specific  Skills  and  Knowledges 

The  results  clearly  indicated  that  students  improved  in  performance 
across  repeated  test  sessions  on  those  tactical  skills  and  knowledges  trained 
in  ARTACT.  Most  of  this  improvement  was  probably  due  to  AOBC  training  itself. 
Nevertheless,  the  strong  impact  of  AOBC  training  on  test  performance  provided 
additional  validation  that  the  skills  and  knowledges  trained  on  ARTACT  are 
integral  to  AOBC.  This  emoirical  finding  is  in  agreement  with  previous 
rational  analyses  (Bryant  tt  al.,  1987)  indicating  that  the  content  of  ARTACT 
includes  integrative,  high  level  skills  and  knowledges  that  span  the  AOBC 
curriculum.  But  the  more  important  finding  was  that  students  in  the 
experimental  group  improved  above  and  beyond  those  students  in  the  control 
group.  The  latter  result  indicated  that  exposure  to  ARTACT  had  a  positive 
effect  on  gains  in  test  performance.  Taken  together,  these  findings  imply 
that  ARTACT  had  a  positive  effect  on  the  student's  preparation  for  their 
subsequent  tactical  training. 

Effects  on  the  Organization  and  Processing  of  Tactical  Knowledges 

The  results  also  showed  that  students  changed  in  the  processing  of  their 
tactical  knowledges  from  a  style  characterized  primarily  by  the  use  of  general 
heuristics  on  the  pretest  to  one  primarily  characterized  by  the  use  of 
specific  knowledge  on  the  posttest.  The  students  showed  an  analogous  change 
in  their  knowledge  organization  evidenced  by  the  increase  in  their  use  of 
doctrinal ly  recognized  schemata  and  in  their  ability  to  interrelate  these  in 
describing  their  understanding  of  planning  tactical  operations.  All  of  these 
changes  were  indicative  of  increasing  expertise  in  tactics.  Although  ARTACT 
students  tended  to  consistently  show  greater  changes  in  improved  organization 
of  knowledge  than  did  the  controls,  these  differences  were  not  statistically 
significant.  Thus,  in  contrast  to  the  results  from  the  knowledge  test,  there 
was  no  clear  evidence  that  ARTACT  had  an  effect  on  the  process  of  gaining 
expertise  greater  than  that  attributable  to  AOBC.  One  conclusion  that  might 
be  drawn  from  these  data  is  that,  although  ARTACT  was  effective  in  training 
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specific  tactical  knowledges,  it  did  not  have  the  desired  effect  on  the 
fundamental  manner  in  which  students  organize  and  process  these  knowledges. 
This  interpretation  is  consistent  with  the  finding  that  accretion  of  expertise 
is  a  relatively  long-term  phenomenon,  not  likely  to  be  greatly  affected  by  the 
short-term  exposure  to  ARTACT.  On  the  other  hand,  one  might  also  speculate 
that  these  cognitive  changes  were  too  subtle  and  the  measures  were  too 
insensitive  to  detect  the  effects  of  ARTACT  over  the  overwhelming  effects  of 
AOBC  on  knowledge  structure  and  process.  The  nonsignificant  trends  in  the 
hypothesized  direction  lend  some  credence  to  the  latter  interpretation. 

Effects  on  Performance  in AOBC 


The  evaluation  also  attempted  to  test  whether  or  not  the  skills  and 
knowledges  learned  in  ARTACT  transfer  to  AOBC  classroom  performance.  Despite 
some  interesting  nonsignificant  trends,  the  results  of  the  analyses  showed  no 
differences  between  experimental  and  control  groups.  Although  class 
performance  measures  possess  obvious  face  validity,  there  are  two  problems 
with  these  criterion  measures.  One  problem  is  that  the  AOBC  measures,  even 
those  taken  from  the  Armor  Performance  Test,  measure  some  skills  and 
knowledges  that  are  not  trained  in  ARTACT.  To  the  extent  that  ARTACT  skills 
and  knowledges  do  not  transfer  to  these  additional  skills,  these  academic 
measures  would  tend  to  underestimate  the  effects  of  ARTACT.  The  second 
problem  is  that  the  AOBC  measures  are  academic  achievement  measures  and,  as 
such,  may  have  psychometric  characteristics  that  are  not  conducive  to 
detecting  group  differences  in  performance.  For  instance,  the  differences  in 
a  few  of  the  measures  may  have  been  obscured  by  ceiling  effects  as  indicated 
by  the  high  mean  values  seen  in  both  groups.  This  criticism  did  not  pertain 
to  the  percentile  rank  measure,  however,  whose  means  were  close  to  the  middle 
of  the  range  (i.e.,  50%).  However,  the  latter  measure  represented  overall 
AOBC  performance,  and  was  (along  with  class  average)  less  relevant  than 
results  from  the  Armor  Performance  Test  for  measuring  ARTACT  skills  and 
knowledges. 


Software  and  Hardware  Problems 


Results  from  the  analysis  of  the  implementation  problems  revealed  a 
variety  of  ARTACT  software  and  hardware  problems.  Given  that  the  present 
research  represented  the  first  implementation  of  ARTACT,  the  presence  of 
program  deficiencies  was  not  unexpected.  Identifying  and  correcting  these 
deficiencies  was  a  goal  from  the  outset.  Consequently,  hardware  and  software 
changes  were  made  during  the  study  in  an  effort  to  improve  the  implementation 
of  the  programs.  Nevertheless,  since  the  Offensive  Operations  and  Defensive 
Operations  modules  continued  to  freeze  occasionally  even  at  the  end  of  the 
study,  it  is  clear  that  the  software  and/or  hardware  deficiencies  have  not  yet 
been  completely  corrected. 

Since  the  programs  could  be  not  be  tested  and  corrected  prior  to  the 
research,  it  is  important  to  take  into  account  the  possible  impact  the 
deficiencies  could  have  on  the  assessment  of  their  overall  effectiveness.  As 
noted  in  the  Results,  all  of  the  programs  except  Planning  FA  Fires  froze  on 
occasion.  When  a  program  froze  prior  to  the  end,  the  program  was  restarted 
from  the  beginning  providing  students  with  additional  training.  Restarting 
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the  program  could  have  made  the  program  appear  more  effective  than  it  actually 
was.  When  a  program  froze  at  the  end,  the  program  was  not  restarted  since  the 
students  had  completed  the  module.  However,  these  students  could  not  receive 
a  printout  of  their  performance.  While  it  is  possible  that  the  failure  to 
receive  a  printout  could  have  reduced  the  effectiveness  of  the  program,  the 
reduction  in  effectiveness  should  have  been  small  since  students  received 
feedback  during  the  program  itself.  Furthermore,  it  is  possible  that  the 
other  students  who  did  receive  a  printout  may  never  have  examined  it  since  the 
printout  was  given  to  each  student  at  the  end  of  a  session.  Unfortunately,  no 
attempt  was  made  to  determine  whether  or  not  students  used  the  printouts. 

Subjective  Reactions 

In  general,  ARTACT  had  no  effect  on  student  perceptions  of  AOBC.  The 
one  exception  to  this  generalization  concerns  their  perceptions  of  the 
difficulty  of  AOBC  topics  that  were  related  to  the  content  of  ARTACT.  These 
topics  were  judged  to  be  more  difficult  by  the  experimental  group  than  by  the 
control  group.  The  two  groups  did  not  differ,  however,  in  their  judgments  of 
the  difficulty  of  AOBC  topics  that  were  unrelated  to  ARTACT.  Since  the 
experimental  group  showed  greater  gains  than  the  control  group  in  knowledges 
and  skills  covered  in  ARTACT,  their  perceptions  of  the  difficulty  of  these 
topics  appear  inconsistent  with  their  performance.  One  possible  explanation 
for  this  somewhat  paradoxical  finding  is  that  the  additional  training  on 
ARTACT  may  have  caused  the  students  in  the  experimental  group  to  be  more 
cognizant  of  the  complexity  of  the  topics.  Although  this  may  be  a  reasonable 
explanation  of  the  finding,  it  is  clearly  ad  hoc  and  cannot  be  tested  from  the 
data  obtained  during  the  evaluation. 

Reactions  of  students  in  the  experimental  group  were  generally  favorable 
toward  ARTACT  becoming  part  of  AOBC,  but  with  qualifications  for  improvements 
they  wanted.  Student  responses  indicated  that  they  had  found  ARTACT  easy  to 
use  and  that  it  had  practical  value  as  training  to  augment  and  better  prepare 
them  for  their  regular  tactical  training.  Planning  FA  Fires  was  perceived  as 
the  most  helpful  program.  This  finding  was  unexpected  since  Planning  FA  Fires 
only  partially  overlaps  the  planning  and  identification  of  Target  Reference 
Points  (TRP)  taught  in  planning  platoon  fires.  However,  equally  surprising 
was  the  fact  that  no  students  perceived  Defensive  Operations  to  be  the  most 
helpful  program.  Defensive  Operations  was  the  last  topic  trained  prior  to  the 
field  exercise  and  compared  to  the  other  three  programs,  contained  the  least 
amount  of  new  knowledge.  Since  the  knowledges  tutored  in  Planning  FA  Fires 
only  partially  overlap  AOBC  training,  it  is  speculated  that  students  judged 
the  helpfulness  of  a  program  on  the  basis  of  the  amount  of  new  information  it 
contained. 

The  presence  of  software  and  hardware  problems  were  found  to  affect 
attitudes  toward  ARTACT,  although  the  direction  of  the  effect  was  unexpected. 
Compared  to  students  who  did  not  experience  any  software  or  hardware  problems, 
students  who  had  to  restart  a  module  rated  ARTACT  as  more  helpful  as  a  way  of 
learning  about  platoon  tactics,  more  effective  in  getting  them  to  think  about 
platoon  tactics,  and  easier  to  use.  They  also  perceived  the  programs  to  be 
more  interesting  and  bestowed  more  praise  upon  the  research  staff.  The  praise 
bestowed  upon  the  research  staff  may  have  been  the  result  of  the  effort  that 
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was  made  by  the  staff  to  ensure  that  each  student  was  provided  an  opportunity 
to  complete  each  ARTACT  program.  The  other  attitudinal  differences  between 
the  students  who  restarted  a  module  and  those  who  did  not  may  be  explained  on 

the  basis  of  cognitive  dissonance  (Festinger,  1957).  The  students  who  had  to 

restart  a  program  may  have  justified  their  efforts  (i.e.,  reduced  cognitive 
dissonance)  by  viewing  ARTACT  more  favorably. 

The  results  also  showed  that  reactions  to  ARTACT  were  related  to  the 
students'  success  in  training  as  indicated  by  their  class  rank  in  AOBC.  The 
higher  ranking  students  tended  to  view  ARTACT  as  a  useful  supplement  to  AOBC, 

and  their  criticisms  were  focused  more  on  improvements  in  feedback.  They 

showed  less  of  a  tendency  than  the  lower  ranking  students  to  be  concerned  with 
the  reliability  of  the  system.  This  finding  suggests  that  the  more  successful 
students  tended  to  react  to  ARTACT 's  most  critical  qualities,  namely  the 
training  benefits  that  it  could  provide  them.  In  contrast,  the  less  success¬ 
ful  students  tended  to  react  to  ARTACT's  superficial  qualities,  specifically 
those  defects  that  could  be  easily  corrected  in  subsequent  versions. 

Conclusions 


In  conclusion,  the  results  of  the  evaluation  indicated  that  ARTACT  has 
merit  as  a  method  for  helping  students  to  learn  tactical  skills  and  knowledges 
in  AOBC.  This  conclusion  is  further  supported  by  the  fact  that  students  were 
generally  positive  in  their  reactions  to  the  system.  However,  two  cautionary 
points  should  al:o  be  mentioned.  The  first  is  that  several  software  problems 
remain  in  the  current  system.  These  technical  problems  are  potentially 
solvable  by  software  engineers  and  do  not  invalidate  ARTACT  as  a  training 
system.  The  second  point  relates  to  the  fact  that,  like  most  Army  programs  of 
instruction,  AOBC  is  a  fluid  curriculum  that  changes  as  doctrine  changes. 

Some  changes  have  already  caused  discrepancies  between  ARTACT  and  AOBC 
instruction.  For  instance,  a  scenario  within  the  ARTACT  module  on  techniques 
of  movement  requires  the  student  to  recall  the  vee  formation  and  indicate 
relative  tank  positions  within  the  platoon.  Students  noted  that  this 
particular  formation  is  no  longer  trained  in  AOBC.  This  sort  of  problem  is 
more  serious  than  software  bugs  in  that  it  potentially  threatens  the  relevancy 
of  ARTACT  instruction.  In  anticipation  of  these  sorts  of  problems,  Los  Alamos 
National  Laboratory  developed  an  authoring  system  for  changing  ARTACT's 
scenario  details,  routes  of  movement,  and  targets  for  indirect  fire.  However, 
the  authoring  system's  capabilities  have  not  been  formally  tested  to  determine 
whether  they  can  be  readily  used  by  training  developers  and  whether  they  are 
sufficiently  comprehensive  to  incorporate  most  doctrinal  changes. 
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APPENDIX  A 


Questionnaires 


A-l.  Student  Background  Questionnaire 


1. 

Name: 

Last 

First 

Middle  Initial 

2. 

Class  Number: 

3. 

Roster  Number: 

4. 

SSN: 

5. 

Date  of  Birth: 

Day  Month  Year 

6. 

Component: 

Active 

Reserve 

National 

Guard 

7. 

Source  of  Commission: 

ROTC 

USMA 

OCS 

Other  (Specify: 

8.  Military  Experience  Prior  to  AOBC? 


No 

Yes  (Describe  below,  including  number  of  years) 


9.  Computer  Experience  (check  all  that  apply): 

_ Played  computer  games 

_ Used  computer  at  school 

_ Used  computer  at  work 

_ Used  computer  at  home 

_ Studied  computer  science 

_ Wrote  computer  programs  using  a  computer  language 

(e.g.,  BASIC,  FORTRAN,  C) 

Other  (Describe:  ) 

PT-57-40 
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A-2.  AOBC/ARTACT  Questionnaire 


Name: 

Date: 

Last 

First 

MI 

Class  Number: 

Roster  Number:_ 

1.  Rate  the  level  of  difficulty  you  experienced  in  understanding  the 
following  AOBC  instruction.  Use  a  scale  of  1  to  100  where  "1"  is 
extremely  easy  and  "100"  is  extremely  difficult. 

_  Engineers  and  Mobility/Counter-Mobility  Operations 

_  Call  For  and  Adjust  Indirect  Fire 

_  Tactical  Air  Operations 

_  NBC  Defensive  Operations 

_  Command  and  Control 

_  Techniques  of  Movement 

_  Cavalry  Operations 

_  Fundamentals  of  Reconnaissance 

_  Fundamentals  of  Security 

_  Fundamentals  of  Offense 

Fundamentals  of  Defense 


2.  How  interesting  was  the  AOBC  classroom  instruction  on  tactics? 


Very  Quite  Somewhat  Slightly  Not 

Interesting  Interesting  Interesting  Interesting  Interesting 

At  All 

3.  How  prepared  were  you  for  applying  tactical  knowledge  during  the  10-day 
war? 


_ •  •  • • 

Very  Quite  Somewhat  SI ightly  Not 

Prepared  Prepared  Prepared  Prepared  Prepared 

At  All 

PT  57-40 
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4.  In  your  opinion,  how  knowledgeable  are  you  in  planning  and  executing 
platoon  tactics? 


Very  Quite  Somewhat  Slightly  Not  Know- 

Knowledgeable  Knowledgeable  Knowledgeable  Knowledgeable  ledgeable 

At  All 

5.  As  a  result  of  having  participated  in  AOBC,  how  prepared  are  you  to  serve 
in  combat  as  the  leader  of  a  tank  platoon? 


Very  Quite  Somewhat  SI ightly  Not 

Prepared  Prepared  Prepared  Prepared  Prepared 

At  All 

6.  How  helpful  were  the  ARTACT  programs  as  a  way  of  learning  about  platoon 
tactics? 


Very  Quite  Somewhat  Slightly  Not  Helpfu 

Helpful  Helpful  Helpful  Helpful  At  All 

7.  How  hard  was  it  to  learn  to  use  the  computer  to  work  on  the  ARTACT 
programs? 

: _ :  :  _ :  _ : _ 

Not  Hard  Slightly  Somewhat  Quite  Very 

At  All  Hard  Hard  Hard  Hard 


8.  How  interesting  were  the  ARTACT  programs? 


Very  Quite  Somewhat  Slightly  Not 

Interesting  Interesting  Interesting  Interesting  Interesting 

At  All 

9.  How  effective  were  the  ARTACT  programs  in  getting  you  to  think  about 
platoon  tactics? 


Very  Quite  Somewhat  Slightly  Not 

Effective  Effective  Effective  Effective  Effective 

At  All 


PT  57-40 


A-3 


10.  Which  ARTACT  program  did  you  find  most  helpful? 

_  Planning  FA  Fires  _  Techniques  of  Movement 

_  Offensive  Operations  _  Defensive  Operations 

11.  What  do  you  like  most  about  the  ARTACT  programs? 


12.  What  do  you  dislike  most  about  the  ARTACT  programs? 


PT  57-40 
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13.  In  your  opinion  should  the  ARTACT  programs  become  part  of  the  AOBC 
curriculum? 

_ Yes 

_ Yes,  but  only  if  they  are  improved  (Answer  question  #14) 

_ No 

14.  What  improvements  or  changes  should  be  made  in  the  ARTACT  programs? 


15.  Use  this  space  to  write  any  other  comments  or  suggestions. 


PT  57-40 
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APPENDIX  B 


Comparisons  of  Experimental  and  Control  Groups 
on  Background  and  Experience  Variables 


Table  B-l 


Experience  Reported  by  Experimental  and  Control  Groups 


Experience 

Group 

Experimental 

Control 

Test 

Statistic 

P 

Military 

M 

1.5 

1.9 

t  (46)  =  0.63 

N.S. 

(In  Years) 

SB 

i  2.3 

1.8 

Reported  Computer 

Experience  in  the 

Following  Categories: 

Games 

f 

21 

22 

(Chi)2  (1)  -  0.58 

N.S. 

% 

84 

81 

School 

f 

19 

18 

(Chi)2  (1)  =  0.55 

N.S. 

% 

76 

67 

Work 

f 

15 

7 

(Chi)2  (1)  =  6.18 

<  .05 

% 

60 

26 

Home 

f 

12 

6 

(Chi)2  (1)  =  3.81 

N.S. 

% 

48 

22 

Computer 

f 

11 

9 

(Chi)2  (1)  =  0.62 

N.S. 

Science 

% 

44 

33 

Programming 

f 

14 

10 

(Chi)2  (1)  =  1.88 

N.S. 

% 

56 

37 

"Other" 

f 

0 

2 

% 

0 

7 
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Table  B-2 


Background  of  Experimental  and  Control  Groups 


APPENDIX  C 


Tactical  Knowledge  Test  Materials 


C.l  Materials  for  Offensive  Scenario 
OPORD  for  the  Offensive  Scenario 


1.  SITUATION 

a.  Enemy  Forces.  Enemy  elements  of  the  7th  Motorized  Rifle  Div.  have 
broken  contact  and  have  withdrawn  to  the  south  and  are  establishing 
hasty  defensive  positions  vie  Elizabethtown.  The  enemy  has  a  Div. 
Security  Zone  consisting  of  a  reinforced  Motorized  Rifle  Bn  in  a 
Combat  Security  role.  Ground  and  air  recons  have  failed  to  locate 
any  elements  of  the  security  zone,  but  we  can  expect  to  encounter 
individual  OPs,  with  company  hasty  defensive  positions  2  to  3  kms 
behind  the  OPs.  The  enemy  is  equipped  with  T-62s  and  BMPs.  He  is 
at  60%  strength  and  has  used  chemical  weapons  within  the  lust  48 
hours. 

b.  Friendly  Forces. 

(1)  TF  1-14  AR  conducts  a  movement  to  contact  to  OBJ  Gold  at  0700 
hours,  1  Jan  1990  to  gain  and  maintain  contact  with  withdrawal 
forces  and  to  destroy  any  enemy  combat  security  detachments  in 
zone.  On  order,  continue  the  attack  to  the  south. 

(2)  Team  C  followed  by  Team  B  is  on  our  right,  to  secure  OBJs  Mine 
and  Rock. 

(3)  TF  1-81  INF  is  on  our  left,  to  secure  OBJ  Hawk. 

(4)  BN  Scouts  will  conduct  a  zone  reconnaissance  with  concentration 
on  Axes  Silver  and  Steel. 

(5)  1-42  FA  is  in  direct  support.  TF  1-14  has  priority. 

(6)  Companies  C  and  D  are  detached  and  the  TF  has  received  Co  C, 
1-81  INF. 

(7)  3d  PLT,  Co  A  is  detached  and  we  have  received  3d  PLT  (mech)  Co 
C,  1-81  INF. 
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2.  MISSION 


Team  A,  TF  1-14  AR  conducts  a  movement  to  contact  along  Axis  Steel  at 
0700  hours  to  PL  Brass  to  support  by  fire  on  OBJs  Rock  and  Ore.  On 
order,  seize  OBJ  Ore.  On  order,  continue  the  attack  south. 

3.  EXECUTION 

a.  Concept  of  the  Operation 

(1)  Manuever:  As  contact  is  possible  south  of  LD  Copper,  the 
company  will  cross  Copper  in  traveling  overwatch  in  a  company 
wedge;  1st  PLT  (Red)  in  the  center,  3d  PLT  mech  (Green)  on  the 
right,  and  2nd  PLT  (White)  on  the  left.  If  contact  is  made 
prior  to  PL  Brass,  the  element  making  contact  will  fix  the 
enemy  and  the  remaining  platoons  will,  on  order,  maneuver  to 
destroy  the  enemy  force.  Upon  arrival  at  PL  Brass,  1st  PLT 
occupy  overwatch  position  vie  CP  7  and  orient  on  OBJ  ORE.  2nd 
PLT  occupy  overwatch  position  vie  CP  11  and  orient  on  OBJ  Rock. 
3d  PLT  (mech),  move  to  CP  9  and  prepare  to  move  to  a  dismount 
point  at  BC  2203,  be  prepared  to  sweep  dismounted  thru  OBJ  Ore. 
1st  and  2nd  PLTs,  on  order,  will  assault  OBJ  Ore  to  CP  5  and  6 
respectively.  The  company  will  consolidate  on  the  OBJ  with 
three  PLTs  abreast.  On  orders,  continue  the  attack  south. 

(2)  Fires: 

(a)  Arty  for  TM  A  is  on  request  only. 

(b)  Smoke  and  DPICM  are  limited. 

(c)  Priority  of  fires  initially  to  1st  PLT. 

(d)  All  TRPs  are  registered  pre-plots. 

(e)  Designated  artillery  targets  as  per  your  overlay. 

b.  Specific  Instructions 

(1)  1st  Platoon  (RED) 

(a)  Lead  element  in  company  wedge,  moving  along  Axis  Steel. 

(b)  Once  contact  is  made,  be  prepared  to  become  base  of  fire 
element. 

(c)  Be  prepared  to  become  an  assault  element. 

(d)  Upon  securing  CP  on  OBJ  Ore,  orient  south  between  TRPs  104 
and  105. 

(2)  2nd  Platoon  (WHITE) 

(a)  Left  element  in  company  wedge,  moving  on  Axis  Steel. 

(b)  Once  contact  is  made,  be  prepared  to  become  base  of  fire 
element. 

(c)  Be  prepared  to  become  an  assault  element. 
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(d)  Upon  securing  CP  6,  orient  south  between  TRPs  104  and  105. 

(e)  Do  not  bypass  any  enemy  resistance. 

(3)  3d  Platoon  (Mech)  (GREEN) 

(a)  Right  element  in  company  wedge,  moving  on  Axis  Steel. 

(b)  Once  contact  is  made,  be  prepared  to  become  base  of  fire 
element. 

(c)  Be  prepared  to  dismount  INF  to  clear. 

(d)  On  orders,  move  to  CP  9  and  prepare  to  assault  BC  2203. 

(e)  Dismount  INF  clear  OBJ  Ore  on  order. 

(f)  Upon  securing  OBJ  Ore,  position  your  platoon  vie  BC  2203, 
orient  south  BC  2204. 

(g)  Do  not  bypass  any  enemy  resistance. 

c.  Coordinating  Instructions 

(1)  ADA  status  is  weapons  tight. 

(2)  M0PP  Level  2  is  in  effect.  Increase  on  orders  or  upon  indirect 
fire. 

(3)  Be  prepared  to  stop  the  enemy  counterattack  on  OBJ  Ore. 

(4)  Report  all  friendly  graphics. 

4.  SERVICE  SUPPORT:  Per  unit  SOP. 

Class  I,  III,  and  V  on  call  thru  1SG. 

5.  COMMAND  AND  SIGNAL 

a.  Signal 

(1)  CE0I  Day  5  in  effect. 

(2)  Standardized  call  signs  are  in  effect. 

(3)  Arty  Fire  Direction  Net  call  sign  is  BIGGUN  ONE.  Frequency  is 
38.15. 

(4)  Radio  listening  silence  is  in  effect  until  LD  Copper. 

b.  Command 

(1)  The  succession  of  command  is  CO,  X0,  1st,  2nd,  3d  platoon 
1 eaders 

(2)  The  CO  will  be  with  1st  PLT  initially. 

(3)  The  X0  will  be  with  3d  PLT  initially. 
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Instructions  for  Administering  the  Offensive  Tactical  Exercise 


The  following  instructions  will  be  prerecorded.  The  experimenter  should  stop 
the  tape  recorder  at  (Pause)  and  restart  at  (Continue). 

IN  THE  FOLLOWING  EXERCISE,  WE  WILL  TEST  YOUR  ABILITY  TO  PLAN  AND  EXECUTE  A 
PLATOON  OPERATION  BY  ASKING  FOR  YOUR  RESPONSES  TO  SPECIFIC  SITUATIONS.  IN 
ADDITION  TO  YOUR  RESPONSES,  WE  ARE  INTERESTED  IN  YOUR  THOUGHTS  AS  YOU  APPROACH 
THESE  SITUATIONS.  IN  ORDER  TO  DO  THIS  WE  WILL  ASK  YOU  TO  TALK  ALOUD  AS  YOU 
WORK  ON  THE  PROBLEMS.  WHAT  WE  MEAN  BY  TALK  ALOUD  IS  THAT  WE  WANT  YOU  TO  SAY 
OUT  LOUD  EVERYTHING  THAT  YOU  MIGHT  SAY  OR  THINK  TO  YOURSELF  SILENTLY.  IF  YOU 
ARE  SILENT  FOR  ANY  LENGTH  OF  TIME  I  WILL  REMIND  YOU  TO  KEEP  TALKING  ALOUD.  IF 
YOU  DON'T  UNDERSTAND  THESE  INSTRUCTIONS,  PLEASE  ASK  THE  EXPERIMENTER  NOW. 

(Pause) 

Answer  any  questions  before  proceeding  to  first  warm-up  problem: 

(Continue) 

BEFORE  WE  TURN  TO  THE  EXERCISE,  WE  WILL  START  WITH  A  PRACTICE  PROBLEM.  WE 
WANT  YOU  TO  TALK  ALOUD  WHILE  YOU  DO  THIS  PROBLEM.  WE  WANT  YOU  TO  RECALL  THE 
FACTORS  OF  METT-T,  WHICH  ARE  USED  TO  ANALYZE  TACTICAL  MISSIONS.  IN  OTHER 
WORDS,  WE  WANT  YOU  TO  TALK  ALOUD  SO  THAT  YOU  INDICATE  WHAT  YOU  ARE  THINKING 
WHILE  YOU  RECALL  EACH  FACTOR.  GO  AHEAD  AND  RECALL  THE  FACTORS  OF  METT-T. 

(Pause) 

Reinforce  almost  any  response  that  approximates  the  correct  answer  (Mission, 
Enemy,  Terrain,  Weather,  Troops,  and  Time)  with  an  encouraging  "GOOD" 

Answer  any  question  before  continuing: 

(Continue) 

NOW  THAT  YOU  UNDERSTAND  THE  INSTRUCTIONS  TO  TALK  ALOUD,  WE  SHALL  PROCEED  WITH 
THE  TACTICAL  EXERCISE.  REMEMBER,  THAT  THIS  IS  A  TEST  AND  I  WILL  NOT  BE  ABLE 
TO  PROVIDE  ANY  FEEDBACK  ON  THE  CORRECTNESS  OF  YOUR  RESPONSES.  BE  SURE  THAT 
YOU  CONTINUE  TO  TALK  ALOUD  AS  YOU  APPROACH  THE  PROBLEMS  PRESENTED  WITHIN  THE 
FOLLOWING  SCENARIO: 

(Pause) 

Answer  any  question  before  proceeding  to  the  tactical  exercise: 
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I.  RECEIVE/ANALYZE  MISSION 


Present  company  level  OPORD  with  appropriate  overlays  to  student,  and 
restart  the  prerecorded  instructions. 

(Continue) 

IN  THE  FOLLOWING  TACTICAL  EXERCISES,  YOU  WILL  ASSUME  THE  ROLE  OF  PLATOON 
LEADER  OF  1ST  PLATOON  (RED)  ATTACHED  TO  COMPANY  TEAM  A  OF  TASK  FORCE  1ST 
OF  THE  14TH  ARMOR...  IT  IS  NOW  0600  HOURS.  YOU  ARE  IN  ATTACK  POSITION 
ALLOY  AWAITING  ORDERS  TO  CONDUCT  A  TACTICAL  OPERATION.  YOU  MAY  MAKE 
NOTES  WHILE  THE  OPORD  IS  BEING  READ,  BUT  ALL  NOTES  WILL  BE  COLLECTED  AT 
THE  END  OF  THE  EXERCISE. 

The  OPORD  is  read  aloud  while  the  student  reads  a  copy  of  the  text.  The 
student  keeps  his  copy  of  the  OPORD  for  reference  throughout  the 
exercise.  He  may  make  notes  on  the  UPORD  or  on  a  separate  piece  of 
paper.  Collect  all  notes  at  the  end  of  the  exercise.  When  the  OPORD  is 
finished,  insert  student  tape  and  prepare  to  record  responses.  The 
instructor  reads  the  remaining  instructions. 

In  this  phase,  you  will  probe  students  for  planning  procedures.  To  the 
extent  possible,  let  student  describe  his  planning  procedures  naturally 
and  in  the  order  he  wishes.  However,  if  he  does  not  provide  a  scheme  of 
manuever  by  the  time  he  is  finished,  probe  him  for  that  information  using 
questions  provided  in  Part  D  below.  Start  recording  student  responses 
and  begin  with  the  following  instructions: 

YOU  ONLY  HAVE  ONE  HOUR  BEFORE  MOVING  TO  THE  LINE  OF  DEPARTURE.  TELL  WHAT 
PLANS  YOU  SHOULD  MAKE  OR  ACTIONS  YOU  SHOULD  TAKE  TO  PREPARE  YOUR  PLATOON 
FOR  THIS  MISSION.  WHAT  TASK  SHOULD  YOU  START  WITH?  PLEASE  REMEMBER  TO 
TALK  ALOUD  AS  YOU  RESPOND. 

(Troop  Leading  Procedures  should  include  (a)  receive  and  analyze  mission, 
(b)  issue  warning  order,  (c)  form  tentative  plan,  (d)  start  necessary 
movement,  (e)  conduct  reconnaissance,  (f)  make  decisions/complete  plan, 
(g)  issue  OPORD,  and  (h)  supervise  and  refine.) 

If  the  student  does  not  know  what  troop  leading  procedures  mean,  you  can 
define  as  "actions  that  a  platoon  leader  takes  prior  to  platoon 
operations. " 

React  to  most  responses  with  a  noncommittal:  UH-HUH,  WHAT  ELSE?  If  the 
student  responds  "movement  plan"  or  "manuever  scheme,"  respond:  THAT’S 
RIGHT,  YOU  WILL  NEED  TO  PLAN  A  SCHEME  OF  MANUEVER.  WHAT  ASPECT  OF  THAT 
SCHEME  DO  YOU  WANT  TO  PLAN  FOR  FIRST?  If  the  student  does  not  respond 
with  manuever  scheme  before  finishing  with  troop  leading  procedures,  you 
will  need  to  prompt:  ONE  ASPECT  OF  PLANNING  THAT  YOU  WILL  NEED  TO 
PROVIDE  IS  A  SCHEME  OF  MANUEVER.  WHAT  ASPECT  OF  THAT  SCHEME  DO  YOU  WISH 
TO  PLAN  FOR  FIRST? 
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Student  may  do  the  following  elements  of  the  manuever  scheme  in  any 
order.  But  he  must  complete  all  three  aspects  before  going  on  to  other 
aspects  of  planning.  In  other  words,  if  the  student  does  not  provide 
this  information,  the  experimenter  will  have  to  elicit  it. 

A.  Select  route.  OK.  USE  A  PENCIL  TO  DRAW  YOUR  CHOICE  OF  ROUTE  ON  THE 
MAP. 

(Drawn  route  should  (a)  start  at  Attack  Position  Alloy,  (b)  use 
SR  261  north  of  LD  Copper,  (c)  stay  within  Axis  Steele,  (d)  bypass 
hills  to  avoid  skylining,  (e)  connect  with  Checkpoint  7,  and 
(f)  terminate  at  Objective  Ore.) 

If  the  other  aspects  of  manuever  have  not  been  covered,  say:  WHAT 
OTHER  ASPECTS  OF  MANUEVER  DO  YOU  NEED  TO  PLAN  FOR? 

B.  Movement  technique/formation.  WHAT  MOVEMENT  TECHNIQUES  WOULD  YOU 
USE  ALONG  THAT  ROUTE?  If  the  student  does  not  break  the  route  into 
two  parts,  you  will  have  to  systematically  ask  the  following 
questions: 

1.  Move  from  attack  position  to  line  of  departure.  Break  up  this 
question  into  separate  questions  concerning  movement  technique 
and  movement  formation. 

a.  OK,  WHAT  MOVEMENT  TECHNIQUE  WOULD  YOU  USE  TO  MOVE  FROM  THE 
ATTACK  POSITION  TO  THE  LINE  OF  DEPARTURE? 

If  the  student  does  not  say  or  describe  the  traveling 
technique,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  TRAVELING  TECHNIQUE. 

b.  WHAT  MOVEMENT  FORMATION  WOULD  YOU  USE  TO  REACH  THE  LINE  OF 
DEPARTURE? 

If  the  student  does  not  say  or  describe  the  column 
formation,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  COLUMN  FORMATION. 

2.  Move  from  line  of  departure  to  overwatch  position.  Break  up 
this  question  into  separate  questions  concerning movement 
technique  and  movement  formation. 

a.  WHAT  MOVEMENT  TECHNIQUE  WOULD  YOU  USE  ONCE  YOU  HAVE  PASSED 
THE  LINE  OF  DEPARTURE? 

If  the  student  does  not  say  or  describe  the  traveling 
technique,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  TRAVELING  TECHNIQUE. 


b.  WHAT  MOVEMENT  FORMATION  WOULD  YOU  USE  TO  REACH  THE 
OVERWATCH  POINT? 

If  the  student  does  not  say  or  describe  the  wedge 
formation,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  WEDGE  FORMATION. 

II.  MOVE  TO  PL  BRASS-SURPRISE  EVENTS 


Before  platoon  reaches  Phase  Line  Brass,  two  surprise  events  occur.  Ask 
the  following  questions,  but  do  not  indicate  the  correct  answer. 

A.  Small  Arms  Fire- -Out  of  Sector 

AS  YOUR  PLATOON  MOVES  ACROSS  PHASE  LINE  TIN,  YOU  RECEIVE  80-100 
ROUNDS  OF  SMALL  ARMS  FIRE  FROM  A  POINT  ON  STATE  ROAD  477,  JUST  SOUTH 
OF  BRAVO  CHARLIE  2225.  HOW  SHOULD  YOU  REACT?  (Provide  contact 
report  and  proceed.) 

B.  Direct  Fire  from  Objective  Ore 

YOU  APPROACHED  PHASE  LINE  BRASS  AND  RECEIVE  YOUR  ORDER  TO  MOVE  INTO 
POSITION  AT  CHECK  POINT  7.  ONCE  IN  POSITION,  YOU  RECEIVE  DIRECT 
FIRE  FROM  IDENTIFIABLE  ENEMY  TANKS  ON  OBJECTIVE  ORE.  HOW  SHOULD  YOU 
REACT?  (Engage  targets  and  provide  contact  report.) 

III.  HASTY  DEFENSE  AT  PL  BRASS 

YOUR  PLATOON  IS  STILL  IN  POSITION  AT  CHECK  POINT  7,  AND  YOU  RECEIVE  THE 
FOLLOWING  FRAGO: 

RED  1,  WHITE  1,  GREEN  1,  THIS  IS  ONE.  PRELIMINARY  REPORTS  FROM  BATTALION 
SCOUTS  INDICATE  TWO  POSSIBLE  COMPANY-SIZE  ENEMY  ARMOR  UNITS  EQUIPPED  WITH 
T-62s  LOCATED  ON  OBJECTIVE  ORE.  COUNTERATTACK  IMMINENT.  IMMEDIATELY 
PREPARE  HASTY  DEFENSE  OF  PRESENT  POSITIONS.  PLAN  INDIRECT  FIRE  TARGET 
LOCATIONS  IN  FRONT  OF  YOUR  POSITIONS  AND  REPORT  TARGET  NOMINATIONS  TO 
FSO.  OVER. 

WHAT  SHOULD  YOUR  FIRST  REACTION  BE  TO  THIS  MESSAGE?  (Authenticate.) 

AS  INDICATED  BY  THE  FRAGO,  YOU  WILL  NEED  TO  PLAN  TARGET  LOCATIONS. 

A.  Tarqet  locations.  Place  the  sheet  showing  target  location  symbols 
in  subject's  view  and  say:  INDICATE  TARGET  LOCATIONS  BY  DRAWING  THE 
SYMBOL  FOR  EITHER  CONVENTIONAL,  LINE,  OR  AREA  TARGETS  AS  APPROPRIATE 
ON  YOUR  MAP. 
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(At  least  one  target  should  be  located  (a)  on  top  of  battle 
position,  (b)  in  front  of  battle  position,  (c)  behind  battle 
position,  (d)  on  avenues  of  approach,  (e)  on  prominent  terrain 
features,  (f)  easily  identifiable  features,  (g)  outside  range  of 
direct  fire  weapons,  and  (h)  outside  of  artificial  boundaries.) 

B.  Shell/fuze  combinations.  Once  they  have  finished  locating  indirect 
fire  target  locations,  say:  NUMBER  THE  TARGETS  CONSECUTIVELY  AND 
INDICATE  APPROPRIATE  SHELL/FUZE  COMBINATIONS  BY  LISTING  EACH  TARGET 
ON  THIS  SHEET  WITH  THE  CORRESPONDING  ACCRONYMS  AS  LISTED  AT  THE 
BOTTOM  OF  THE  SHEET.  (Not  scored.) 

IV.  ASSAULT  OF  OBJ  ORE 

THE  ENEMY  DOES  COUNTERATTACK  BUT  SUFFERS  HEAVY  LOSSES.  YOU  RECEIVE 
ANOTHER  MESSAGE  FROM  THE  COMPANY  COMMANDER  TO  CONTINUE  THE  MISSION  TO 
SECURE  CHECK  POINTS  5  AND  6  AND  BE  PREPARED  TO  ASSAULT  BRAVO  CHARLIE 
2203. 


A.  Movement .  If  student  has  not  already  done  so,  have  him  plan  the 
movement  from  CP  7  to  CP  5,  including 

1.  Select  route.  USE  A  PENCIL  TO  INDICATE  ON  THE  MAP  YOUR  CHOICE 
OF  ROUTE  FROM  YOUR  PRESENT  POSITION  TO  THE  OBJECTIVE. 

If  the  other  aspects  of  manuever  have  not  been  covered,  say: 
WHAT  OTHER  ASPECTS  OF  MANUEVER  DO  YOU  NEED  TO  PLAN  FOR? 

2.  Movement  formation.  WHAT  MOVEMENT  FORMATION  WOULD  YOU  USE  TO 
REACH  THE  OBJECTIVE? 

If  the  student  does  not  say  or  describe  the  line  formation, 
read  the  following  statement:  THE  DOCTRINAL  CHOICE  FOR  THIS 
SITUATION  IS  THE  LINE  FORMATION. 

B.  Surprise  Event--Indirect  Fire. 

AS  YOU  APPROACH  THE  OBJECTIVE,  ARTILLERY  BEGINS  BURSTING  ALL  AROUND 
YOU.  IT  APPEARS  TO  BE  HIGH  EXPLOSIVES  MIXED  WITH  SMOKE.  WHAT 
SHOULD  YOU  DO?  (Don  masks,  button  up,  go  faster.) 

V.  CONCLUSION 

YOU  SURVIVE  THE  INDIRECT  FIRE  ATTACK  AND  YOU  HAVE  SECURED  THE  OBJECTIVE. 
THE  PLATOON  REPORTS  A  WINGMAN  TANK  THAT  HAS  SUFFERED  ONE  WOUNDED  IN 
ACTION  REQUIRING  EVACUATION.  THE  PLATOON  HAS  EXPENDED  30  SABOT  ROUNDS, 
12  HEAT  ROUNDS,  3000  COAX  ROUNDS,  500  CAL  .50  ROUNDS,  AND  1000  GALLONS 
DIESEL.  WHAT  ACTION  SHOULD  YOU  TAKE?  (Send  a  SITREP  (Blue-2).) 
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VI.  ADDITIONAL  QUESTIONS 


A.  Draw  Formations. 


1.  Column. 

EARLIER  IN  THE  SCENARIO  WE  DISCUSSED  THE  COLUMN  FORMATION. 
PLEASE  DRAW  THE  COLUMN  FORMATION  INDICATING  THE  POSITIONS 
OF  ALL  FOUR  TANKS  AND  THEIR  GUN  TUBE  ORIENTATIONS. 

2.  Wedge. 

DO  THE  SAME  THING  FOR  THE  WEDGE  FORMATION. 

3.  Line. 


DO  THE  SAME  THING  FOR  THE  LINE  FORMATION. 

B.  Hand-and-arm  signals.  Present  hand  and  arm  signals  with  letters 
identifying  eacn.  Ask  for  the  signals  by  letter  for  the  following 
techniques  or  formations: 


1. 

2. 

3. 

4. 


Hand 

Hand 

Hand 

Hand 


and  arm  signal  for  traveling  technique, 
and  arm  signal  for  column  formation, 
and  arm  signal  for  the  wedge  formation, 
and  arm  signal  for  the  line  formation. 
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C-2.  Materials  for  Defensive  Scenario 
OPORD  for  the  Defensive  Scenario 


1.  SITUATION 

a.  Enemy  Forces.  We  can  expect  elements  of  the  143rd  Motorized  Rifle 
Division  to  attack  our  defensive  positions  with  a  reinforced 
Motorized  Rifle  Brigade,  with  two  Motorized  Rifle  Companies  leading, 
one  following.  The  S-2  indicated  that  the  Brigade  is  at  90% 
strength  and  are  equipped  with  T-62s,  BMPs,  and  BRDMs.  The  enemy 
has  used  nonpersistent  chemical  agents  in  the  last  24  hours. 

b.  Friendly  Forces. 

(1)  TF  1-14  defends  in  sector  NLT  H-HR,  on  order  CATK  to  complete 
the  destruction  of  the  enemy. 

(2)  Bn  Set  PLT  is  presently  conducting  screening  operations,  in 
front  of  the  TF,  vie  the  98  E-W  grid  line,  and  on  order  they 
will  shift  to  a  subsequent  screen  on  the  east  flank  of  the  TF. 

(3)  TM  B  is  defending  BP  75  NLT  H-HR  on  our  left  flank,  and  on 
order  will  displace  to  BP  33. 

(4)  TM  C  is  defending  BP  41  NLT  H-HR  on  our  right  flank,  and  on 
order  will  displace  to  BP  57. 

(5)  TM  D  is  defending  BP  82  NLT  H-HR,  and  on  order  will  displace  to 
BP  25. 

(5)  1-40  FA  is  in  direct  support  of  the  Brigade. 

c.  Attachments/Detachments .  We  have  the  3/C/2-87  Mech  attached. 

2.  MISSION 

TM  A  defends  BP  62  NLT  H-HR.  On  order,  displace  to  BP  28.  On  order, 

counterattack  to  complete  enemy  destruction. 

3.  EXECUTION 

a.  Concept  of  the  Operation 
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(1)  Manuever:  TM  A  will  conduct  a  deliberate  occupation  of  BP  62 
to  be  established  NLT  H-HR,  oriented  on  EA  Eagle.  The  Bn  Scout 
PLT  will  provide  early  warning  of  the  enemy's  attack,  and  on 
order,  they  will  shift  to  the  east  flank  of  the  TF.  Engage 
targets  outside  EA  eagle  with  indirect  fire  only.  Engage 
targets  inside  EA  Eagle  with  direct  fires  only.  The  trigger 
point  will  be  when  a  company-sized  element  enters  EA  Eagle. 

The  breakpoint  will  be  when  four  or  more  enemy  vehicles  breach 
or  bypass  the  minefield  in  EA  eagle.  If  the  enemy  main  attack 
is  from  our  east  flank,  we  will  shift  to  supplementary  BPs  or 
displace  and  conduct  a  hasty  occupation  of  BP  28.  On  order,  I 
will  issue  a  FRAGO  to  counterattack  and  re-establish  BP  62. 

(2)  Fires:  Priority  of  fires  within  the  team  will  go  to  1st  PLT. 
All  TRPs  are  also  registered  artillery  preplots.  Platoon 
leaders  be  prepared  to  plan  defensive  indirect  targets  as 
required.  Submit  indirect  fire  target  nominations  to  company 
FSO. 

(3)  Obstacles:  Engineer  support  has  already  completed  all 
obstacles.  See  overlay. 

(4)  Specific  instructions:  See  execution  matrix. 

(5)  Coordinating  instructions:  MOPP  Level  2  is  now  in  effect. 
Upgrade  on  order  or  upon  receiving  indirect  fire. 

4.  SERVICE  AND  SUPPORT:  By  unit  SOP. 

5.  COMMAND  AND  SIGNAL 
a.  Command. 

(1)  I  will  be  with  2nd  PLT. 

(2)  XO  will  be  with  the  3rd  PLT. 

(3)  Succession  of  command:  XO,  1st,  2nd,  3rd,  and  MECH  PLT 
leaders. 


Signal . 

(1) 

Day 

5  of  the 

CEOI 

is 

in  effect. 

(2) 

1st 

PLT 

call 

sign 

is 

Red-1. 

(3) 

2nd 

PLT 

call 

sign 

is 

White-1. 

(4) 

3rd 

PLT 

cal  1 

sign 

is 

Blue-1. 
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(5)  Mech  FLT  call  sign  is  Green-1. 

(6)  TM  CDR  call  sign  is  per  CEOI. 

(7)  FIST  call  sign  is  Black  1. 

(8)  Artillery  Fire  Direction  Net  Call  sign  is  X72,  frequency  31.25. 


Instructions  for  Administering  the  Defensive  Tactical  Exercise 


The  following  instructions  will  be  prerecorded.  The  experimenter  should  stop 
the  tape  recorder  at  (Pause)  and  restart  at  (Continue). 

IN  THE  FOLLOWING  EXERCISE,  WE  WILL  TEST  YOUR  ABILITY  TO  PLAN  AND  EXECUTE  A 
PLATOON  OPERATION  BY  ASKING  FOR  YOUR  RESPONSES  TO  SPECIFIC  SITUATIONS.  IN 
ADDITION  TO  YOUR  RESPONSES,  WE  ARE  INTERESTED  IN  YOUR  THOUGHTS  AS  YOU  APPROACH 
THESE  SITUATIONS.  IN  ORDER  TO  DO  THIS  WE  WILL  ASK  YOU  TO  TALK  ALOUD  AS  YOU 
WORK  ON  THE  PROBLEMS.  WHAT  WE  MEAN  BY  TALK  ALOUD  IS  THAT  WE  WANT  YOU  TO  SAY 
OUT  LOUD  EVERYTHING  THAT  YOU  MIGHT  SAY  OR  THINK  TO  YOURSELF  SILENTLY.  IF  YOU 
ARE  SILENT  FOR  ANY  LENGTH  OF  TIME  I  WILL  REMIND  YOU  TO  KEEP  TALKING  ALOUD.  IF 
YOU  DON'T  UNDERSTAND  THESE  INSTRUCTIONS,  PLEASE  ASK  THE  EXPERIMENTER  NOW. 

(Pause) 

Answer  any  questions  before  proceeding  to  first  warm-up  problem: 

(Continue) 

BEFORE  WE  TURN  TO  THE  EXERCISE,  WE  WILL  START  WITH  A  PRACTICE  PROBLEM.  WE 
WANT  YOU  TO  TALK  ALOUD  WHILE  YOU  DO  THIS  PROBLEM.  WE  WANT  YOU  TO  RECALL  THE 
FACTORS  OF  METT-T,  WHICH  ARE  USED  TO  ANALYZE  TACTICAL  MISSIONS.  IN  OTHER 
WORDS,  WE  WANT  YOU  TO  TALK  ALOUD  SO  THAT  YOU  INDICATE  WHAT  YOU  ARE  THINKING 
WHILE  YOU  RECALL  EACH  FACTOR.  GO  AHEAD  AND  RECALL  THE  FACTORS  OF  METT-T. 

(Pause) 

Reinforce  almost  any  response  that  approximates  the  correct  answer  (Mission, 
Enemy,  Terrain,  Weather,  Troops,  and  Time)  with  an  encouraging  "GOOD" 

Answer  any  question  before  continuing: 

(Continue) 
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NOW  THAT  YOU  UNDERSTAND  THE  INSTRUCTIONS  TO  TALK  ALOUD,  WE  SHALL  PROCEED  WITH 
THE  TACTICAL  EXERCISE.  REMEMBER,  THAT  THIS  IS  A  TEST  AND  I  WILL  NOT  BE  ABLE 
TO  PROVIDE  ANY  FEEDBACK  ON  THE  CORRECTNESS  OF  YOUR  RESPONSES.  BE  SURE  THAT 
YOU  CONTINUE  TO  TALK  ALOUD  AS  YOU  APPROACH  THE  PROBLEMS  PRESENTED  WITHIN  THE 
FOLLOWING  SCENARIO: 

(Pause) 

Answer  any  question  before  proceeding  to  the  tactical  exercise: 

I.  RECEIVE/ANALYZE  MISSION 


Present  company  level  OPORD  with  appropriate  overlays  to  student,  and 
restart  the  prerecorded  instructions. 

(Continue) 

IN  THE  FOLLOWING  TACTICAL  EXERCISES,  YOU  WILL  ASSUME  THE  ROLE  OF  PLATOON 
LEADER  OF  1ST  PLATOON  (RED)  ATTACHED  TO  COMPANY  TEAM  A  OF  TASK  FORCE  1ST 
OF  THE  14TH  ARMOR...  IT  IS  NOW  0600  HOURS  WITH  H-HOUR  SET  FOR  0700.  YOU 
ARE  IN  ASSEMBLY  AREA  ALPHA  AWAITING  ORDERS  TO  CONDUCT  A  TACTICAL 
OPERATION.  YOU  MAY  MAKE  NOTES  WHILE  THE  OPORD  IS  BEING  READ,  BUT  ALL 
NOTES  WILL  BE  COLLECTED  AT  THE  END  OF  THE  EXERCISE. 

The  OPORD  is  read  aloud  while  the  student  reads  a  copy  of  the  text.  The 
student  keeps  his  copy  of  the  OPORD  for  reference  throughout  the 
exercise.  He  may  make  notes  on  the  OPORD  or  on  a  separate  piece  of 
paper.  Collect  all  notes  at  the  end  of  the  exercise.  When  the  OPORD  is 
finished,  insert  student  tape  and  prepare  to  record  responses.  The 
instructor  reads  the  remaining  instructions. 

In  this  phase,  you  will  probe  students  for  planning  procedures.  To  the 
extent  possible,  let  student  describe  his  planning  procedures  naturally 
and  in  the  order  he  wishes.  However,  if  he  does  not  provide  a  scheme  of 
manuever  by  the  time  he  is  finished,  probe  him  for  that  information  using 
questions  provided  in  Part  D  below.  Start  recording  student  responses 
and  begin  with  the  following  instructions: 

YOU  ONLY  HAVE  ONE  HOUR  BEFORE  MOVING  TO  YOUR  BATTLE  POSITION.  TELL  WHAT 
PLANS  YOU  SHOULD  MAKE  OR  ACTIONS  YOU  SHOULD  TAKE  TO  PREPARE  YOUR  PLATOON 
FOR  THIS  MISSION.  WHAT  TASK  SHOULD  YOU  START  WITH?  PLEASE  REMEMBER  TO 
TALK  ALOUD  AS  YOU  RESPOND. 

(Troop  Leading  Procedures  should  include  (a)  receive  and  analyze  mission, 
(b)  issue  warning  order,  (c)  form  tentative  plan,  (d)  start  necessary 
movement,  (e)  conduct  reconnaissance,  (f)  make  decisions/complete  plan, 
(g)  issue  OPORD,  and  (h)  supervise  and  refine.) 
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II.  MOVE  TO  BATTLE  POSITION  62  DELTA 


If  the  student  has  not  spontaneously  provided  a  manuever  scheme  before 
finishing  describing  his  plans,  you  will  need  to  prompt:  ONE  ASPECT  OF 
PLANNING  THAT  YOU  WILL  NEED  TO  PROVIDE  IS  A  SCHEME  OF  MANUEVER.  WHAT 
ASPECT  OF  THAT  SCHEME  DO  YOU  WISH  TO  PLAN  FOR  FIRST? 

Student  may  do  the  following  elements  of  the  manuever  scheme  in  any 
order.  But  he  must  complete  all  three  aspects  before  going  on  to  other 
aspects  of  planning.  In  other  words,  if  the  student  does  not  provide 
this  information,  the  experimenter  will  have  to  elicit  it. 

A.  Select  route.  OK,  USE  A  PENCIL  TO  DRAW  YOUR  CHOICE  OF  ROUTE  ON  THE 
MAP. 

If  the  other  aspects  of  manuever  have  not  been  covered,  say:  WHAT 
OTHER  ASPECTS  OF  MANUEVER  DO  YOU  NEED  TO  PLAN  FOR? 

(Drawn  route  should  (a)  start  at  Assembly  Area  Alpha,  (b)  use 
indicated  route  between  Checkpoints  7  and  8,  (c)  avoid  planned 
barricades  and  minefields,  (d)  bypass  hills  to  avoid  skylining,  and 
(e)  terminate  at  BP  62D.) 

B.  Movement  technique/formation.  WHAT  MOVEMENT  TECHNIQUES  WOULD  YOU 
USE  ALONG  THAT  ROUTE?  If  the  student  does  not  break  the  route  into 
two  parts,  you  will  have  to  systematically  ask  the  following 
questions: 

1.  Move  from  Assembly  Area  Alpha  to  the  battle  position.  Break  up 
this  question  into  separate  questions  concerning  movement 
technique  and  movement  formation. 

a.  OK,  WHAT  MOVEMENT  TECHNIQUE  WOULD  YOU  USE  TO  MOVE  FROM  THE 
ASSEMBLY  AREA  TO  THE  BATTLE  POSITION? 

If  the  student  does  not  say  or  describe  the  traveling 
technique,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  TRAVELING  TECHNIQUE. 

b.  WHAT  MOVEMENT  FORMATION  WOULD  YOU  USE  WHILE  ON  THIS  ROUTE? 

If  the  student  does  not  say  or  describe  the  column 
formation,  read  the  following  statement:  THE  DOCTRINAL 
CHOICE  FOR  THIS  SITUATION  IS  THE  COLUMN  FORMATION. 

2.  At  battle  position.  If  the  student  has  not  provided  a  stop 
formation  as  part  of  his  plan,  ask  the  following: 

OK,  WHAT  MOVEMENT  FORMATION  WOULD  YOU  USE  ONCE  YOU  HAVE  REACHED 
THE  BATTLE  POSITION? 
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If  the  student  does  not  say  or  describe  the  coil  formation, 
read  the  following  statement:  THE  DOCTRINAL  CHOICE  FOR  THIS 
SITUATION  IS  THE  COIL  FORMATION. 

III.  PLAN  INDIRECT  FIRE  TARGET  LOCATIONS  AT  BATTLE  POSITION  62  DELTA 


If  the  student  has  not  already  planned  indirect  fires,  have  him  do  so 
now. 

AS  INDICATED  IN  THE  OPERATIONS  ORDER,  YOU  NEED  TO  BE  PREPARED  TO  NOMINATE 
INDIRECT  FIRE  TARGET  LOCATIONS. 

A.  Tarqet  locations.  Place  the  sheet  showing  target  location  symbols 
in  subject’s  view  and  say:  INDICATE  TARGET  LOCATIONS  BY  DRAWING  THE 
SYMBOL  FOR  EITHER  CONVENTIONAL,  LINE,  OR  AREA  TARGETS  AS  APPROPRIATE 
ON  YOUR  MAP.  (Not  scored.) 

B.  Shell/fuze  combinations.  Once  they  have  finished  locating  indirect 
fire  target  locations,  say:  NUMBER  THE  TARGETS  CONSECUTIVELY  AND 
INDICATE  APPROPRIATE  SHELL/FUZE  COMBINATIONS  BY  LISTING  EACH  TARGET 
ON  THIS  SHEET  WITH  THE  CORRESPONDING  ACRONYMS  AS  LISTED  AT  THE 
BOTTOM  OF  THE  SHEET. 

(At  least  one  target  should  be  located  (a)  on  top  of  battle 
position,  (b)  in  front  of  battle  position,  (c)  behind  battle 
position,  (d)  on  avenues  of  approach,  (e)  on  prominent  terrain 
features,  (f)  easily  identifiable  features,  (g)  outside  range  of 
direct  fire  weapons,  and  (h)  outside  of  artificial  boundaries.) 

IV.  TARGET  ON  STATE  ROAD  261 


A.  Tanks  outside  of  Engagement  Area  Eagle.  YOU  HAVE  SPOTTED  AN 
IDENTIFIABLE  ENEMY  TANK  PLATOON  ON  STATE  ROAD  261  IN  THE  VICINITY  OF 
GRID  LOCATION  607965  (point).  WHAT  ACTION  SHOULD  YOU  TAKE?  (Call 
for  indirect  fire  North  .3  of  TRP  207) 

B.  Possible  HIND-D.  THE  ENEMY  ATTACK  HAS  BEEN  TEMPORARILY  SLOWED,  BUT 
IN  THE  MEAN  TIME,  YOU  SPOT  A  HIND-D  ON  THE  HORIZON.  HE  EVIDENTLY 
HAS  NOT  SEEN  YOU  OR  YOUR  PLATOON.  WHAT  ACTION  SHOULD  YOU  TAKE? 

(Go  into  hide  position  and  provide  report.) 

C.  Tanks  within  Engagement  Area  Eagle.  THE  ENEMY  CONTINUES  TO  ADVANCE 
IN  LARGER  NUMBERS.  YOU  SPOT  A  COMPANY-SIZED  ENEMY  ARMOR  UNIT  ON 
STATE  ROAD  261  IN  THE  VICINITY  OF  GRID  592952  (point).  WHAT  ACTION 
SHOULD  YOU  TAKE?  (Issue  platoon  fire  command/engage  targets  with 
direct  f ire. ) 
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V.  HASTY  OCCUPATION  OF  BATTLE  POSITION  28 


A.  Move  from  Battle  Position  62  Delta.  TWO  COMPANY-SIZED  ARMOR  UNITS 
ARE  NOW  IN  ENGAGEMENT  AREA  EAGLE  AND  HAVE  STARTED  TO  BYPASS 
MINEFIELD  AND  HEAD  TOWARD  YOUR  POSITION.  THE  COMPANY  COMMANDER 
ISSUES  A  FRAGO  TO  MOVE  QUICKLY  TO  BATTLE  POSITION  28  AND  BEWARE  OF 
ENEMY  IN  ZONE.  WHAT  IS  YOUR  SCHEME  OF  MANUEVER  FOR  GETTING  THERE? 

As  before,  provide  the  following  prompts  only  if  needed. 

1.  Select  route.  Provide  red  pencil  and  prompt: 

INDICATE  YOUR  CHOICE  OF  ROUTE  FROM  YOUR  PRESENT  BATTLE  POSITION 
TO  THE  SUPPLEMENTARY  BATTLE  POSITION  USING  THIS  PENCIL. 

(Route  should  (a)  go  behind  BPs  62  and  28,  and  (b)  terminate  at 
BP  28C. 

2.  Select  movement  technique.  OK,  WHAT  MOVEMENT  TECHNIQUE  WOULD 
YOU  USE  TO  MOVE  TO  THE  SUPPLEMENTARY  BATTLE  POSITION? 

If  the  student  does  not  say  or  describe  the  traveling  overwatch 
technique,  read  the  following  statement:  THE  DOCTRINAL  CHOICE 
FOR  THIS  SITUATION  IS  TRAVELING  OVERWATCH. 

3.  Select  movement  formation.  WHAT  MOVEMENT  FORMATION  WOULD  YOU 
USE  WHILE  ON  THIS  ROUTE? 

If  the  student  does  not  say  or  describe  the  column  formation, 
read  the  following  statement:  THE  DOCTRINAL  CHOICE  FOR  THIS 
SITUATION  IS  THE  COLUMN  FORMATION. 

B.  Sagger  Missile  from  Right  Flank.  AS  YOUR  PLATOON  MOVES  TO  BATTLE 
POSITION  28,  A  BMP  ADVANCES  AROUND  BATTLE  POSITION  62  DELTA  AND 
FIRES  A  SAGGER  MISSILE  AT  YOUR  PLATOON.  WHAT  IS  YOUR  ACTION? 

(Alert  platoon,  conduct  evasive  action  drill  and  report.) 

C.  FRAGO.  YOU  RECEIVE  THE  FOLLOWING  FRAGO  OVER  THE  RADIO: 

RED-1,  THIS  IS  ONE.  WHITE  HAS  ATTRITTED  TO  25%  STRENGTH.  OCCUPY 
BATTLE  POSITION  28  CHARLIE.  CONSOLIDATE  WHITE  PLATOON'S  FIRES  WITH 
YOURS.  ORIENT  BETWEEN  TRPs  215  to  219.  OVER. 

WHAT  IS  YOUR  FIRST  REACTION  TO  THIS  MESSAGE?  (Authenticate.) 

VI.  SURPRISE  EVENT— INDIRECT  FIRE 

AS  YOU  APPROACH  THE  OBJECTIVE,  ARTILLERY  BEGINS  BURSTING  ALL  AROUND  YOU. 
IT  APPEARS  TO  BE  HIGH  EXPLOSIVES  MIXED  WITH  SMOKE.  WHAT  SHOULD  YOU  DO? 
(Don  masks,  button  up,  go  faster.) 
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VII.  CONCLUSION 

ONCE  ON  BATTLE  POSITION  28  CHARLIE,  THE  COMPANY  IS  ATTACKED  FROM  THE 
DIRECTION  OF  BATTLE  POSITION  62.  YOUR  PLATOON  SUCCESSFULLY  DEFENDS  THE 
SUPPLEMENTARY  POSITION  BUT  SUFFERS  MOBILITY  KILLS  TO  TWO  TANKS  AND  TWO 
WOUNDED  IN  ACTION,  BOTH  REQUIRING  EVACUATION.  YOUR  PLATOON  SERGEANT 
REPORTS  THAT  YOU'VE  EXPENDED  50  ROUNDS  OF  SABOT,  13  ROUNDS  OF  HEAT,  2000 
COAX  ROUNDS,  200  ROUNDS  OF  CAL  .50  AMMUNITION,  AND  1200  GALLONS  OF  FUEL. 
WHAT  ACTION  SHOULD  YOU  TAKE?  (Send  SITREP  (Blue-2).) 


VIII.  ADDITIONAL  QUESTIONS 


A.  Draw  Formations. 


1.  Column. 

EARLIER  IN  THE  SCENARIO  WE  DISCUSSED  THE  COLUMN  FORMATION. 
PLEASE  DRAW  THE  COLUMN  FORMATION  INDICATING  THE  POSITIONS 
OF  ALL  FOUR  TANKS  AND  THEIR  GUN  TUBE  ORIENTATIONS. 

2.  Coil. 

DO  THE  SAME  THING  FOR  THE  COIL  FORMATION. 

B.  Hand-and-arm  signals.  Present  hand  and  arm  signals  with  letters 
identifying  each.  Ask  for  the  signals  by  letter  for  the  following 
techniques  or  formations: 

1.  Hand  and  arm  signal  for  traveling  technique. 

2.  Hand  and  arm  signal  for  traveling  overwatch  technique. 

3.  Hand  and  arm  signal  for  the  column  formation. 

4.  Hand  and  arm  signal  for  the  coil  formation. 
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APPENDIX  D 


Intercorrelations  Among  Tactical  Knowledge  Measures 


Table  D-l 

Intercorrelations  Among  Knowledge  Variables  for  Both  Experimental  and 
Control  Groups  ( N  =  51) 


Where  Measured 

Pretest 

Armor 

Performance 

Test 

Posttest 

Variable 

1  2  3 

4 

5 

6 

7 

8 

9 

10 

Pretest 

1 .  Knowledge  Test 

1.000  -0.405“  -0.394“ 

-0.287* 

-0.081  ' 

■0.303* 

0.153 

-0.272 

-0.378“ 

0.326* 

2.  SK  Incidental  Verba  1 Izat >on> 

1.000  0.293* 

0.103 

0.123 

0.121 

-0.072 

0.323* 

0.329* 

-0.108 

3.  GH  Incidental  Verbalizations 

1.000 

0.175 

-0.165 

0.139 

-0.163 

0.375“ 

0.472“ 

0.019 

4.  Use  of  Schemata 

1.000 

0.043  ‘ 

■0.065 

-0.075 

0.153 

0.132 

0.237 

Armor  Performance  Test 

S.  Performance  Component 

1.000 

0.269 

-0.137 

0.238 

-0.052 

-0.100 

6.  Written  Component 

1.000 

-0.252 

0.256 

0.132 

-0.152 

Posttest 

7.  Knowledge  Test 

1.000 

-0.229 

-0.265 

0.094 

8.  SK  Incidental  Verbalizations 

1.000 

0.261 

-0.125 

9.  GH  Incidental  Verbalizations 

1.000 

-0.333* 

10.  Use  of  Schemata 

1.000 

Mote.  Knowledge  Test  score  Is  the  average  error  rate  computed  for  the  test  objectives.  SK  and  GH  Incidental  Verbalizations 
refer  to  comments  made  by  students  on  the  knowledge  test  Indicating  the  use  of  specific  knowledge  and  general  heuristics, 
respectively.  Use  of  Schemata  Is  a  binary  variable  Indicating  whether  or  not  students  used  doctrlnally  recognized  schemata 
to  organize  their  responses  to  the  CHALLENGE  program. 


Table  D-2 


Intercorrelations  Among  Knowledge  Variables  for  Experimental  Group 
( N  =25) 


Where  Measured 

Pretest 

Armor 

Performance 

Test 

Posttest 

Variable 

1  2 

3 

4 

6 

1 

8 

9 

10 

Pretest 

1.  Knowledge  Test 

I. 000  -0.316 

-0.573“ 

-0.247 

0.096  -0.361 

0.150 

-0.472* 

-0.463* 

0.250 

2.  SK  Incidental  Verbalizations 

1.000 

0.2S6 

0.209 

0.114 

0.108 

-3.169 

0.184 

0.545“ 

0.175 

3.  GH  Incidental  Verbalizations 

1.000 

0.119 

-0.138 

0.240 

•0.058 

0.443“ 

0.712“ 

0.062 

4.  Use  of  Schemata 

1.000 

0.055 

0.008 

0.015 

0.238 

0.121 

0.327 

Armor  Performance  Test 

5.  Performance  Component 

1.000 

0.344 

-0.117 

0.064 

-0.304 

0.122 

6.  Written  Component 

1.000 

-0.148 

0.300 

0.224 

-0.146 

Posttest 

7.  Knowledge  Test 

1.000 

-0.255 

-0.056 

-0.230 

8.  SK  Incidental  Verbalizations 

1.000 

0.348 

-0.154 

9.  GH  Incidental  Verbalizations 

1.000 

-0.047 

10.  Use  of  Schemata 

1.000 

Mote.  Knowledge  Test  score  Is  the  average  error  rate  for  the  test  objectives.  SK  and  6H  Incidental  Verbalizations  refer  to 
connents  made  by  students  on  the  knowledge  test  Indicating  the  use  of  specific  knowledge  and  general  heuristics,  respectively. 
Use  of  Schemata  Is  a  binary  variable  Indicating  whether  or  not  students  used  doctrlnally  recognized  schemata  to  organize 
their  responses  to  the  CHALLENGE  program. 


Intercorrelations  Among  Knowledge  Variables  for  Control  Group  ( N  =  26) 


Where  Measured 

Pretest 

Armor 

Performance 

Test 

Posttest 

Variable 

1  2 

3 

4 

5 

7 

8 

9 

10 

Pretest 

1.  Knowledge  Test 

1.000  -0.423* 

-0.193 

-0.280 

•0.350  ' 

-0.256 

0.299 

-0.111 

-0.345 

0.329 

2.  SK  Incidental  Verbalizations 

1.000 

0.373 

-0.038 

0.210 

0.149 

-0.134 

0.443* 

0.211 

-0.219 

3.  SH  Incidental  Verbalizations 

1.000 

0.298 

-0.240  ‘ 

-0.047 

-0.282 

0.321 

0.281 

-0.051 

4.  Use  of  Schemata 

1.000 

0.077  • 

-0.149 

-0.208 

0.086 

0.143 

0.243 

Armor  Performance  Test 

5.  Performance  Component 

1.000 

0.181 

-0.091 

0.400* 

0.132 

-0.341 

6.  Written  Component 

1.000 

-0.374 

0.218 

0.059 

-0.175 

Posttest 

7.  Knowledge  Test 

1.000 

-0.232 

-0.381 

0.362 

S.  SK  Incidental  Verbalizations 

1.000 

0.209 

-0.112 

9.  GH  Incidental  Verbalizations 

1.000 

-0.501** 

10.  Use  of  Schemata 

1.000 

Mote.  Knowledge  Test  score  Is  the  average  error  rate  for  all  test  objectives.  $K  and  6H  Incidental  Verbalizations  refer  to 
coments  nade  by  students  on  the  knowledge  test  Indicating  the  use  of  specific  knowledge  and  general  heuristics,  respectively. 
Use  of  Schemata  Is  a  binary  variable  Indicating  whether  or  not  students  used  doctrlnally  recognized  schemata  to  organize  their 
responses  to  the  CHALLENGE  program. 


APPENDIX  E 

Analysis  of  Variance  Summary  Tables 

Table  E-l 

Summary  of  Analysis  of  Errors  on  Tactical  Test 


Group  (GRP) 

1 

0.00 

0.00 

Order  (ORD) 

1 

0.12 

1.64 

GRP  X  ORD 

1 

0.18 

2.46 

Error,, 

47 

0.08 

Test  Objective  (OBJ) 

6 

0.65 

16.00*** 

6,  42 

0.32 

14.66' 

GRP  X  OBJ 

6 

0.08 

1.87 

6,  42 

0.76 

2.25 

ORD  X  OBJ 

6 

0.09 

2.20 

6,  42 

0.76 

2.18 

GRP  X  ORD  X  OBJ 

6 

0.05 

1.18 

6,  42 

0.85 

1.24 

Error* 

282 

0.04 

Trials  (TRL) 

1 

123.91 

344.34*** 

GRP  X  TRL 

1 

2.94 

8.16** 

ORD  X  TRL 

1 

0.12 

0.34 

GRP  X  ORD  X  TRL 

1 

0.36 

0.10 

Error* 

47 

0.04 

OBJ  X  TRL 

6 

0.74 

18.83*** 

6,  42 

0.28 

18.36' 

GRP  X  OBJ  X  TRL 

6 

0.04 

0.88 

6,  42 

0.89 

0.86 

ORD  X  OBJ  X  TRL 

6 

0.23 

5.84*** 

6,  42 

0.55 

5.72' 

GRP  X  ORD  X  OBJ  X  TRL 

6 

0.04 

1.10 

6,  42 

0.85 

1.25 

Error* 

282 

0.04 

*p  <  .05 
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Table  E-2 


Summary  of  Analyses  of  Student  Comments  Related  to  Different  Levels  of 
Knowledge 


Source 

df 

MS 

F 

Group  (GRP) 

1 

Specific  Knowledge 

5.144 

0.75 

Errorb 

49 

33.711 

Trials  (TRL) 

1 

14.486 

4.08* 

GRP  X  TRL 

1 

5.074 

1.43 

Errorw 

49 

17.377 

Group  (GRP) 

1 

General  Heuristics 

0.473 

0.05 

Errorb 

49 

9.060 

Trials  (TRL) 

1 

15.785 

4.37* 

GRP  X  TRL 

1 

0.451 

0.12 

Error* 

49 

3.609 

Group  (GRP) 

1 

Intermediate  Level 

0.015 

0.03 

Errorb 

49 

0.527 

Trials  (TRL) 

1 

0.157 

0.28 

GRP  X  TRL 

1 

0.000 

0.00 

Error* 

49 

0.568 

LO 

o  o 

V  V 

•V 

E-2 

Table  E-3 

Summary  of  Analysis  of  Confidence  on  Challenge 


Group  (GRP) 

1 

1216.30 

4.69* 

Error,, 

49 

259.16 

Trials  (TRL) 

1 

2150.28 

14.32** 

GRP  X  TRL 

1 

148.71 

0.99 

Error* 

49 

150.13 

<  .05 

<  .01 
<  .001 
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Table  E-4 


Summary  of  Analysis  of  Schemata  Usage 


Source 

df 

MS 

F 

Presence  of 

Schemata 

Group  (GRP) 

1 

0.02 

0.06 

Error,, 

50 

0.27 

Trials  (TRL) 

1 

2.87 

17.60*** 

GRP  X  TRL 

1 

0.56 

3.46 

Error* 

50 

0.16 

Number  of  Instances 

of  Schemata  Usage 

Group  (GRP) 

1 

0.38 

0.52 

Error,, 

50 

0.73 

Trials  (TRL) 

1 

5.90 

11.00** 

GRP  X  TRL 

1 

1.52 

2.84 

Error* 

50 

0.54 

‘p  <  .05 

„  P  <  • 01 
p  <  .001 
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Table  E-5 


Summary  of  Analyses  of  Overall  AOBC  Measures 


Source 

df 

MS 

Class  Average 

Group  (GRP) 

1 

23.96 

Errorb 

56 

4.82 

Percentile  Rank 

Group  (GRP)  1  2888.28 

Errorb  56  742.36 


<  .05 

<  .01 
<  .001 


2.72 


3.89 
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Table  E-6 


Summary  of  Analysis  of  Differences  Between  Written  and  Performance  Components 
of  the  Armor  Performance  Test 


Source 

df 

MS 

F 

Group  (GRP) 

1 

435.31 

1.63 

Error,, 

56 

266.96 

Component  (CO) 

1 

568.02 

3.66 

GRP  X  CO 

1 

125.12 

0.81 

Error* 

56 

154.85 

<  .05 

<  .01 

<  .001 
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Table  E-7 


Summary  of  Analysis  of  Differences  Between  Objectives  Within  Written  and 
Performance  Components  of  the  Armor  Performance  Test 


ANOVA 

MANOVA 

Source 

df 

MS 

F 

df 

Lambda 

F 

Written  Component 

Group  (GRP) 

3 

207.89 

0.45 

Errorb 

56 

458.56 

Objective  (OBJ) 

1 

16.27 

0.04 

GRP  X  OBJ 

1 

54.25 

0.13 

Errorw 

56 

419.33 

Performance  Component 

Group  (GRP) 

1 

2115.08 

1.67 

Errorb 

56 

1267.19 

Objective  (OBJ) 

3 

1713.70 

2.08 

3,  54 

0.90 

2.04 

GRP  X  OBJ 

3 

329.09 

0.40 

3,  54 

0.98 

0.38 

Error* 

56 

823.56 

‘p  <  .05 

**p  <  .01 
"*  p  <  .001 
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Table  E-8 


Summary  of  Analysis  of  Difficulty  Ratings  of  AOBC  Topics 


ANOVA 

MANOVA 

Source 

df 

MS 

F 

df 

Lambda 

F 

Group  (GRP) 

1 

2933.05 

0.69 

Errorb 

38 

4277.09 

Topic  (TOP) 

10 

3416.66 

7.96** 

10, 

29 

0.31 

6.42** 

Related  TOP  (REL) 

5 

1436.56 

5.46** 

5, 

34 

0.56 

5.37** 

Unrelated  TOP  (UNR) 

4 

6685.12 

12.58** 

4, 

35 

0.43 

11.69** 

REL  vs.  UNR  (R/U) 

1 

243.38 

0.29 

GRP  X  TOP 

10 

783.05 

1.82 

10, 

29 

0.62 

1.76 

GRP  X  REL 

5 

404.92 

1.54 

5. 

34 

0.88 

0.90 

GRP  X  UNR 

4 

579.76 

1.09 

4, 

52 

0.85 

1.54 

GRP  X  R/U 

1 

3486.81 

4.10* 

Errorw 

380 

428.98 

Error,, 

190 

262.84 

Error,, 

152 

531.49 

Error,3 

38 

849.66 

>  <  -05 

p  <  .01 
***  -  _ 


E-8 


Table  E-9 


Summary  of  Analysis  of  Differences  Between  Items  Relating  to  AOBC 


Source 

ANOVA 

MAN0VA 

df 

MS 

F 

df 

Lambda 

F 

Group  (GRP) 

1 

0.70 

0.78 

Errorb 

38 

0.90 

Question  (QSN) 

3 

0.21 

0.54 

3,  36 

0.95 

0.69 

GRP  X  QSN 

3 

0.10 

0.26 

3,  36 

0.97 

0.41 

Error* 

114 

0.38 

*p  <  .05 

*>  <  .01 

***  nn 
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Table  E-10 


Summary  of  Analysis  of  Items  Relating  to  ARTACT  as  a  Function  of  Problems 
Experienced  During  the  Training  Sessions 


Source 


_ ANQVA 

17  MS 


MANOVA 

Lambda 


Problems  (Y/N) 

1 

15.68 

Error,, 

23 

3.54 

Question  (QSN) 

3 

5.66 

Y/N  X  QSN 

3 

0.16 

Error* 

69 

0.71 

4.43* 

8.01***  3,  21  0.55  5.71“ 

0.22  3,  21  0.95  0.38 


<  .05 

<  .01 
<  .001 
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APPENDIX  F 


Content  Analysis  of  Incidental  Verbalizations 


The  instructions  to  talk  aloud  while  answering  questions  on  the  tactical 
knowledge  test  were  relatively  successful  in  evoking  verbalizations  that  were 
incidental  in  that  they  were  above  and  beyond  their  direct  answers  to 
questions.  The  student  transcripts  were  examined  to  reveal  commonalities 
among  these  verbal  data.  This  content  analysis  indicated  that  10  different 
types  of  verbalizations  were  given  by  at  least  two  students  on  either  the 
prestest  or  the  posttest.  Most  of  the  verbalizations  provided  at  least  a 
partial  rationale  for  the  students'  answers.  However,  other  types  of 
verbalizations  were  noted  as  well.  Each  type  of  comment  is  defined  below  with 
verbatim  examples  taken  from  st-uent  transcripts.  The  mean  frequency  of 
occurrence  for  each  type  of  verbalization  is  displayed  in  Table  E-l. 

1.  In  explaining  the  reason  for  a  response,  students  often  said  it  was 
dictated  by  a  specific  characteristic  of  the  situation  or  the  action 
itself.  This  sort  of  comment  often  is  in  the  form  of  an  M if-then" 
rule,  which  does  not  make  sense  to  those  who  are  not  knowledgeable 
in  tactics. 

o  "We'd  be  moving  in  traveling,  since  I'm  being  overwatched." 

o  "If  he  can  cover  my  move  with  another  platoon,  then  I  can  use  the 
fastest  means  possible  to  get  to  my  other  battle  position,  which 
would  be  traveling. . .If  he  can't  cover  my  move,  I'd  have  to  bound 
in  section." 

2.  Another  reason  for  a  particular  action  (or  nonaction)  was  that  the 
student  wanted  to  achieve  some  desirable  outcome  or  avoid  a 
nondesirable  one.  This  reasoning  appears  to  be  based  on  common 
sense  or  deduction  that  should  make  more  sense  to  those  who  are  less 
knowledgeable  in  tactics. 

o  "I  don't  want  to  fire  at  it.  It's  almost  impossible  for  a  tank 
to  hit  one  of  those  vehicles..." 

o  "I  would  probably  want  ONE  to  authenticate  (to)  make  sure  that 
it's  not  bogus  information. 

3.  Students  sometimes  provided  a  statement  of  some  relevant  general 
principle(s)  that  potentially  guides  responses  to  tactical  scenario, 
but  is  nottied  to  any  particular  answer  within  a  scenario. 

o  "...I'd  want  to  designate  the  technique  of  movement  and  a 
formation.  That  would  be  based  on  likely  enemy  contact." 

o  "Definitely  need  to  use  cover  and  concealed  routes.  Stay  way 
from  the  built-up  areas.  Stay  out  of  open  areas." 
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4.  Instead  of  telling  us  why  he  did  something,  the  student  sometimes 
spontaneously  told  us  how  he  does  something.  (Does  not  count 
initial  planning  where~Re  was  specifically  asked  how  to  plan  for  an 
operation. ) 

o  "I  would  then  give  my  platoon  a  platoon  fire  command... I  would 
give  them  AT  MY  COMMAND.  What  I  would  say  would  be  RED,  THIS  IS 
RED  ONE,  TANK  DIRECT  FRONT,  CROSS  AT  MY  COMMAND,  OUT." 

o  "I  could  do  a  deliberate  (occupation)  in  an  hour.  First  thing  we 
would  do  is  stop  in  back  of  Battle  Position  D  on  the  back  side  of 
this  hill  in  a  coil...." 

5.  Students  sometimes  reported  that  they  responded  on  the  basis  of 
anticipated  enemy  actions. 

o  "First  of  all,  I'd  put  (an  indirect  fire  target  location) 
here... It's  a  good  place  for  the  enemy  to  get  good  location. 

o  "I'd  probably  want  to  put  (a  target  location)  right  on  top  of 

them.  And  I  would  assume  they're  ...  on  the  hilltop." 

6.  Student  sometimes  explicitly  stated  that  information  in  the  orders 

(OPORD  or  FRAGO)  was  the  reason  for  a  specific  response. 

o  "(The)  OP  order  says  we'd  be  the  base  of  fire,  so  I'd  give  a 

contact  to  my  platoon,  give  them  an  action  drill,  and  begin 
firing  on  whoever  is  firing  on  us." 

o  "It  looks  like  he's  expecting  the  attack  from  the  east,  because 
that's  the  only  one  he  mentioned  in  the  OP  order... I'd  go  with 
that  (indirect  fire  target  location)." 

7.  If  the  student  explicitly  indicated  that  the  reason  for  a  particular 
response  was  his  interpretation  of  the  mission,  his  comment  fell  in 
this  category. 

o  "We're  the  defense,  but  we're  the  hasty  defense  right  now,  but 
we're  still  going  down,  so  I  don't  want  to  drop  anything  that 
won't  explode  on  catact,  so  I'll  probably  go  with  the  HE 
(she 11 /fuze  combination)." 

o  "...if  there  was  time,  I  might  call  some  indirect  fire  down  on 
battle  position  62.  But  our  objective  there  is  get  to  our 
subsequent  battle  position,  so  I  would  (just)  return  fire." 

8.  Students  sometimes  commented  that  their  responses  were  based  on 
doctrine  or  SOP  or  other  rote  tactical  rules  without  further 
elaboration. 
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o  "What  I'd  do  first  is  first  thing  give  a  contact  report.  SOP." 

o  (What  movement  formation  would  you  use  along  this  route?) 
"Doctrine  indicates  column." 

9.  Students  sometimes  based  their  responses  on  a  personal  preference. 

o  "There  may  be  too  many  (indirect  fire  target  locations),  but  I 
like  to  overemphasize  artillery." 

o  "I  like  to  stay  on  the  left  (side  of  the  wedge  formation).  I 
think  the  doctrine  is  stay  on  the  right,  but  I'm  left-handed.  So 
I'd  usually  have  my  tank  on  the  left  side." 

10.  Not  often  students  provided  a  mnemonic  or  imaginal  aid  that  they 
use  to  remember  the  correct  response  to  a  question. 

o  (What  is  the  hand/arm  signal  for  the  traveling  overwatch 
technique?)  "The  bird.  The  bird  is  K." 

o  "The  column  is  basically  waving  your  hand  in  front  of  your  face 
like  waving  to  a  parade." 
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Table  F-l 


Mean  Number  of  Verbalizations  by  Category 


As  Measured  At 


Type  of  Comments 

Group 

Pretest 

Posttest 

Specific 

Experimental 

3.08 

4.12 

Characteristic 

Control 

3.81 

4.08 

Anticipated 

Experimental 

3.40 

2.72 

Outcome 

Control 

2.96 

2.69 

General 

Experimental 

0.60 

1.08 

Principles 

Control 

1.27 

0.88 

Interpretation  of 

Experimental 

0.64 

0.80 

0P0RD/FRAG0 

Control 

0.81 

0.85 

Enemy  Actions 

Experimental 

0.60 

0.36 

Control 

0.77 

0.38 

"How-To" 

Experimental 

0.16 

0.60 

Information 

Control 

0.04 

1.23 

Required  by 

Experimental 

0.28 

0.28 

Mission 

Control 

F-4 

0.23 

0.08 

(table  continues) 

As  Measured  At 


Type  of  Comments 

Group 

Pretest 

Posttest 

Dictated  by 

Experimental 

0.04 

0.12 

Doctrine/SOP 

Control 

G.12 

0.35 

Personal 

Experimental 

0.08 

0.08 

Preference 

Control 

0.04 

0.31 

Mnemonic/Imaginal 

Experimental 

0.04 

0.16 

Coding 

Control 

0.15 

0.08 
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APPENDIX  G 


Additional  Analyses  of  Unstructured  Questionnaire  Items 


Introduction 


Responses  to  the  four  open-ended  questionnaire  items  were  discussed 
briefly  in  the  Results  ruction  of  this  report.  A  more  detailed  description 
of  the  responses  and  breakdowns  by  uncontrolled  events  and  student 
characteristics  are  presented  below. 


Description  of  Responses 

Most  liked  aspects.  The  responses  made  to  the  question  asking  students 
what  they  liked  most  about  the  ARTACT  programs  are  shown  in  Table  G-l.  A 
total  of  35  responses  were  given  to  the  question,  and  only  two  students  did 
not  answer  it.  The  responses  were  sorted  into  clusters  with  six  clusters 
emerging.  The  largest  cluster,  which  contained  ten  responses  (made  by  nine 
students),  consisted  of  statements  praising  ARTACT' s  role  in  helping  to 
prepare  for  AOBC  classes.  The  students  claimed,  for  example,  that  ARTACT 
helped  them  to  prepare  for  the  field  exercise  or  that  it  provided  them  an 
opportunity  to  practice  their  skills.  The  next  largest  cluster,  which 
contained  eight  responses  (made  by  five  students),  praised  specific 
characteristics  of  the  ARTACT  programs.  The  students  stated,  for  example, 
that  all  of  the  subjects  came  together  in  ARTACT  or  that  the  graphics  were 
interesting.  Another  cluster,  which  contained  seven  responses  (made  by  six 
students),  focused  on  feedback,  e.g.,  the  comment  that  they  were  able  to 
participate  in  the  exercise  and  then  receive  feedback  on  their  performance. 

The  remaining  clusters  focused  on  the  specific  information  that  was  learned 
(e.g.,  hand-and-arm  signals,  field  artillery),  remarked  on  the  competence  of 
the  research  staff,  or  contained  miscellaneous  comments. 

Most  disliked  aspects.  The  responses  made  to  the  question  asking 
students  what  they  disliked  most  about  the  ARTACT  programs  are  shown  in 
Table  G-2.  A  total  of  46  responses  were  given  to  the  question,  and  only  two 
students  failed  to  answer  it.  The  responses  were  sorted  with  seven  clusters 
emerging.  The  largest  cluster,  which  contained  ten  responses  (made  by  six 
students),  consisted  of  statements  that  were  critical  of  the  ARTACT  training 
schedule.  The  students  claimed,  for  example,  that  the  ARTACT  programs  took  up 
their  free  time,  took  time  away  from  training,  and  were  not  coordinated  with 
their  AOB  classes.  Two  clusters  contained  nine  responses.  One  of  these 
pertained  to  criticisms  of  specific  aspects  of  the  program  (made  by  six 
students).  One  student,  for  example,  stated  that  the  programs  were  redundant 
at  times,  while  another  did  not  like  the  emphasis  on  formations  and  movement 
techniques.  One  student  claimed  that  ARTACT  required  an  existing  knowledge  of 
tactics,  while  another  felt  that  the  program  moved  ahead  even  when  the  student 
was  not  ready.  The  other  cluster  containing  nine  responses  (made  by  eight 
students)  criticized  ARTACT  for  deviating  from  doctrine.  Other  clusters 
contained  responses  criticizing  the  rigidity  with  which  ARTACT  accepted 
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answers  (e.g.,  the  claim  that  only  certain  phrases  were  accepted  by  the 
computer  or  the  observation  that  answers  could  not  be  changed),  the  quality  of 
the  map  display,  and  the  equipment  problems  that  occurred  during  the 
evaluation.  The  final  cluster  consisted  of  three  miscellaneous  responses. 

Recommended  improvements.  The  responses  made  to  the  question  asking  what 
improvements  or  changes  should  be  made  in  ARTACT  are  shown  in  Table  G-3.  A 
total  of  46  responses  were  given  to  the  question,  although  three  students  did 
not  answer  it.  Nine  clusters  emerged  when  the  responses  were  sorted.  The 
largest  cluster,  which  contained  ten  responses  (made  by  six  students), 
contained  recommendations  concerning  the  characteristics  of  the  programs. 
Recommendations  were  made,  for  example,  that  the  programs  be  more  flexible  in 
accepting  answers,  that  deviations  from  doctrine  be  allowed,  and  that  the 
amount  of  instructional  detail  be  increased.  The  next  largest  cluster,  which 
consisted  of  seven  responses  (made  by  six  students),  contained  recommendations 
that  the  software  and  hardware  be  improved.  The  remaining  clusters  focused  on 
the  map  display  (including  the  recommendation  that  paper  maps  be  used), 
feedback  (including  the  recommendation  that  an  evaluator  be  present  to  discuss 
each  student's  performance),  scenarios  and  missions,  consistency  with 
classroom  instruction  and  doctrine,  uses  for  ARTACT  (e.g.,  to  teach  offensive 
and  defensive  missions,  to  teach  the  basics),  and  scheduling  (e.g.,  the 
recommendation  that  ARTACT  be  available  during  classroom  hours).  Finally,  one 
additional  cluster  contained  two  miscellaneous  responses. 

Comments  and  suggestions.  The  responses  made  to  the  question  asking 
students  to  write  any  other  comments  or  suggestions  are  shown  in  Table  G-4.  A 
total  of  32  responses  were  made  to  the  item,  although  11  students  did  not 
respond  to  it.  Four  clusters  emerged  when  the  responses  were  sorted.  The 
largest  cluster,  which  contained  13  responses  (made  by  five  students), 
contained  a  diversity  of  critical  comments.  Among  them  were  the  suggestions 
that  a  programmed  text  be  used  instead  of  ARTACT  and  that  ARTACT  not  become 
part  of  the  AOBC  curriculum.  One  response  was  simply  the  statement  that 
ARTACT  training  was  disappointing.  On  the  other  hand,  there  was  a  cluster  of 
five  responses  (made  by  four  students)  that  were  complimentary.  Among  them 
were  the  statements  that  the  programs  were  enjoyable,  that  all  AOBC  students 
should  participate  on  ARTACT,  and  that  ARTACT  provided  an  outstanding  learning 
experience.  Another  cluster  contained  eight  miscellaneous  suggestions  (made 
by  seven  students)  including  the  suggestion  that  the  student  be  "killed"  when 
making  a  drastic  mistake,  that  tactics  instructors  examine  the  programs,  and 
that  the  AOBC  students  who  use  the  M60  tank  train  on  ARTACT  while  the  other 
AOBC  students  train  on  U-COFT.  The  final  cluster  contained  six  responses 
(made  by  four  students)  that  praised  the  research  staff. 

Correlations  with  Uncontrolled  Events  or  Student  Characteristics 


Analyses  were  performed  to  determine  if  the  responses  to  the  four  open- 
ended  questions  correlated  with  the  hardware  and  software  problems  that  many 
of  the  students  experienced  (as  indicated  by  having  to  restart  at  least  one  of 
the  four  ARTACT  modules)  or  with  the  students'  success  in  AOBC  (as  indicated 
by  rank  in  AOBC  class  and  amount  of  improvement  on  the  tactical  knowledge 
test).  To  perform  these  analyses,  the  first  step  was  to  identify  the 
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characteristics  of  the  students  who  made  each  response  to  each  question. 

Every  response  was  categorized  according  to  whether  or  not  the  student  who 
made  it  experienced  a  hardware/ software  problem,  was  in  the  top  or  bottom  half 
of  his  AOBC  class,  and  was  above  or  below  the  median  in  improvement  on  the 
tactical  knowledge  test.  The  number  of  students  within  each  category  who  made 
each  response  are  shown  in  Tables  6-1  through  6-4.  The  next  step  was  to  count 
the  number  of  students  who  made  at  least  one  response  within  each  response 
cluster.  This  information  is  also  shown  in  Tables  6-1  through  6-4.  To 
determine  if  the  responses  correlated  to  each  of  the  three  student 
characteristics,  a  set  of  2  x  2  tables  was  created.  Each  table  contained 
cross-tabulations  based  on  characteristics  of  the  students  and  the  responses 
made  within  a  cluster.  That  is,  the  number  of  students  making  at  least  one 
response  within  a  cluster  and  the  number  of  students  not  making  at  least  one 
response  were  determined  for  students  with  the  characteristic  being  examined 
and  again  for  those  without  it.  Since  the  numbers  of  students  responding 
within  individual  clusters  was  small,  Fisher's  Exact  Tests  were  conducted  to 
determine  whether  or  not  the  correlations  between  student  characteristics  and 
questionnaire  responses  were  significant.  The  results  are  summarized  below. 

Correlations  with  hardware/software  problems.  Having  a  hardware  or 
software  problem  did  not  correlate  significantly  with  responses  to  the  items 
asking  students  what  they  liked  most  about  the  ARTACT  programs,  what  they 
disliked  most  about  the  programs,  or  what  improvements  or  changes  should  be 
made  in  the  ARTACT  programs.  However,  experiencing  a  hardware  or  software 
problem  did  relate  significantly  to  comments  on  ARTACT  staffmembers  made  on 
the  questionnaire  item  asking  students  to  write  other  comments  or  suggestions. 
Four  of  the  ten  students  who  experienced  a  software  or  hardware  problem 
praised  the  research  staff  compared  to  none  of  the  fifteen  students  who  did 
not  experience  such  a  problem,  p  -  .017.  There  was  also  a  greater  trend  for 
the  students  who  experienced  a  problem  to  respond  to  this  item,  although  the 
trend  was  not  statistically  significant:  Two  of  the  10  students  experiencing 
a  hardware/ software  problem  failed  to  respond  to  this  question,  while  9  of  the 
15  students  who  did  not  experience  a  problem  failed  to  respond,  p  =  .099. 

Correlations  with  class  rank  in  AOBC.  Rank  in  AOBC  correlated  signif i- 
cantly  with  responses  made  on  just  one  of  the  four  open-ended  questions:  On 
the  item  asking  what  improvements  or  changes  should  be  made  in  the  ARTACT 
programs,  four  of  the  thirteen  students  in  the  top  half  of  their  class 
recommended  changes  or  improvements  in  feedback  compared  to  none  of  the  twelve 
students  in  the  bottom  half  of  their  class,  p  -  .039.  Two  other  correlations 
with  class  rank  approached  statistical  significance:  (a)  On  the  same 
questionnaire  item,  just  one  student  in  the  top  half  of  his  class  recommended 
the  use  of  more  reliable  software  or  hardware  compared  to  five  students  in  the 
bottom  half  of  their  class,  p  -  .073;  and  (b)  on  the  questionnaire  item  asking 
students  what  they  liked  most  about  the  ARTACT  programs,  seven  students  in  the 
top  half  of  their  class  cited  ARTACT' s  role  in  helping  them  to  prepare  for 
AOBC  compared  to  just  two  students  in  the  bottom  half,  p  =  .097. 

Correlations  with  improvement  in  tactical  knowledge  test  scores.  Amount 
of  improvement  from  the  pretest  to  the  posttest  on  the  tactical  knowledge  test 
correlated  only  with  responses  to  the  question  asking  students  what 
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improvements  or  changes  should  be  made  in  the  ARTACT  program.  Four  of  the 
twelve  students  whose  improvement  in  tactical  knowledge  was  below  the  median 
recommended  improvements  in  the  ARTACT' s  map  display  while  none  of  the 
thirteen  students  whose  improvement  was  above  the  median  made  this  suggestion, 
p  =  .039.  There  was  also  a  nonsignificant  trend  for  students  in  the  lower 
half  of  knowledge  improvement  to  propose  additional  uses  for  ARTACT,  p  =  .096. 
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Table  G-l 


Number  of  Responses  to  ARTACT  Questionnaire  Item  Asking  Students  What 
They  Liked  Most  About  ARTACT  Proqrams  (Presented  by  Response  Cluster 
and  Event/Student  Characteristic) 


Response  Cluster 


What  Student  Liked  Host 


_ _ Event/Student  Characteristic _ 

Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number 

of  Responses  No  Yes  High  Low  High  Low 


Preparation  for  AOBC  Classes 


Provided  preparation  for  armor 
exam  and  field  problem 

Program  was  helpful  during  the 
10-day  war 

Provided  additional  Instruction 

Provided  a  chance  to  practice 
skills 

Provided  preparation  for  AOB 
Instruction 

Provided  exposure  to  platoon 
operations  before  classroom 
Instruction 

Using  tactics  training  before 
10-day  war 

Provided  preparation  for  later 
classes 


1  1 


Programs  complement  their 
counterparts 

Helpful  addition  to  classroom 
Instruction 

(Number  of  Students 
Responding  to  Cluster) 


1  1 


(7)  (2)  (7)  (2)  (3)  (6) 


(table  continues) 


Event/Student  Characteristic 


Response  Cluster 


What  Student  Uked  Most 


Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number 

of  Responses  No  Yes  High  Low  High  Low 


(table  continues) 
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Event/Student  Characteristic 


Response  Cluster 
What  Student  Liked  Most 


Experienced 
Equipment 
Total  Problems 

Number 

of  Responses  No  Yes 


Improvement 
In  Tactical 
Class  Rank  Knowledge 

High  Low  High  Low 


Feedback 

Going  through  the  exercise 
alone 

1 

1 

1 

1 

Getting  Inmedlate  feedback 

2 

2 

1 

1 

1  1 

Seeing  what  I  know  and  do  not 
know 

1 

1 

1 

1 

Learning  by  running  a  mission 
and  being  given  the  solution 

1 

1 

1 

1 

Getting  to  plan  offense  and 
defense,  and  seeing  the 
mistakes 

1 

1 

1 

1 

Applying  knowledge  and  getting 
instant  feedback 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(6) 

(2) 

(4) 

(2) 

(4} 

(5)  (1) 

(table  continues) 
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Event /Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledae 

What  Student  Liked  Host 

Number 
of  Responses 

No 

Yes 

High 

Low 

High 

Low 

Information  Learned 

Hand-and-arm  signals,  techniques 
of  movement,  troop  leading  procedures 

1 

1 

1 

1 

Field  artillery 

1 

1 

1 

1 

Introduction  to  planning  and  offense 

1 

1 

1 

1 

Primer  for  tactics 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(3) 

(1) 

(2) 

(2) 

(2) 

(2) 

Research  Personnel 

Mr.  Foster 

1 

1 

1 

1 

Competent  people  running  ARTACT 
program 

1 

1 

1 

1 

Very  good  staff 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(3) 

(1) 

(2) 

(1) 

(2) 

(1) 

(2) 

Miscellaneous 

Better  than  terrain  boards 

1 

1 

1 

1 

Good  location 

1 

1 

1 

1 

Getting  to  the  end 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(3) 

(2) 

(1) 

(1) 

(2) 

(2) 

(1) 

No  Comnents 

(Number  of  Students) 

(2) 

(1) 

(1) 

(1) 

(1) 

(0) 

(2) 
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Table  G-2 

Number  of  Responses  to  ARTACT  Questionnaire  Item  Asking  Students  What 
They  Most  Disliked  About  ARTACT  Programs  (Presented  by  Response 
Cluster  and  Event/Student  Characteristic) 


Response  Cluster 

What  Student  Disliked  Host 


_ Event/Student  Characteristic 

Experienced  Irrprovement 

Equipment  in  Tactical 

Total  Problems  Class  Rank  Knowledge 

Hunger 

of  Responses  No  Yes  High  Low  High  Low 


Scheduling 

ARTACT  program  came  before 

classroom  instruction  1 

Equipment  problems  made  schedule 
inconvenient  2 

Not  coordinated  with  AOB  classes  1 

Uses  up  free  time  2 

Had  to  wait  at  least  30  minutes 

before  starting  1 

Instruction  took  too  much  time 

away  from  training  I 

ARTACT  should  be  conducted  during 

training  hours  1 

ARTACT  schedule  conflicted  with  school  1 

(Nuirtber  of  Students 

Responding  to  Cluster)  (6) 


1 

2  1 
1 
2 

1 

1  1 

1 

1 

(4)  (2)  (2) 


1  1 

1  1  1 

1  1 

2  1  I 

1  1 

1 

1  1 

1  1 

(4)  (3)  (3) 


(table  continues) 
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Response  Cluster 

What  Student  Disliked  Host 


_ Event/Student  Characteristic 

Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number 

of  Responses  No  Yes  High  Low  High  Low 


Differences  from  Ooctrlne 


Programs  presented  definite  solutions; 
some  were  wrong 

1 

1 

1 

1 

Did  not  agree  with  solutions  pertaining 
to  "techniques  of  movement" 

1 

1 

1 

1 

The  requirements  presented  for  good  route 
selection  contradicted  the  school 
solution  that  was  presented 

1 

1 

1 

1 

There  were  contradictions  within  the 
programs 

1 

1 

1 

1 

Programs  frequently  differed  from 
classroom  Instruction 

3 

2 

1 

1 

2 

2 

1 

Mismatch  between  doctrine  and  school 

1 

1 

1 

1 

Programs  did  not  show  the  school 
way  or  field  way 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(8) 

(5) 

(3) 

(4) 

(4) 

(4) 

(4) 

(table 

continues) 
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_ Event/Student  Characteristic 

Experienced  Inproveaent 

Response  Cluster  Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number 

What  Student  Disliked  Host  of  Responses  No  Yes  High  Low  High  Low 


Specific  Aspects  of  the  Program 


The  beginning  of  the  program  with 
the  Colonel 

1 

1 

1 

1 

Showing  the  Captain 

1 

1 

1 

1 

Showing  the  movement  of  the  tracks 

1 

1 

1 

1 

Too  much  concentration  on  formations 
movement  techniques 

1 

1 

1 

1 

ARTACT  assumed  knowledge  of  tactics 

1 

1 

1 

1 

Program  moved  ahead  even  when  the 
student  was  not  ready 

1 

1 

1 

1 

Evaluation  matrix  was  nonsensical 

1 

1 

1 

1 

Programs  were  redundant  at  times 

1 

1 

1 

1 

Did  not  point  out  better  choices 
and  show  why 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(6) 

(4) 

(2) 

(4) 

(2) 

(2) 

(4) 

(table 

continues) 
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Event/Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledae 

Number 

What  Student  Disliked  Host  of  Responses 

No 

Yes 

High 

Low 

High  Low 

Riqidity  of  Proqrams 

Program  was  rigid— had  to  match  school 
solution 

2 

2 

1 

1 

2 

Program  was  more  rigid  than  instructors 

1 

1 

1 

1 

Could  not  vary  from  route  selected  by 
school 

1 

1 

1 

1 

The  constraints  on  movement  were  slightly 
rigid 

1 

1 

1 

1 

Cooputer  accepted  only  certain  phrases 

1 

1 

1 

1 

Program  looks  for  key  words 

1 

1 

1 

1 

Program  was  rigid— couldn't  change  answers 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(7) 

(5) 

(2) 

(4) 

(3) 

(2)  (5) 

Quality  of  Map  Display 

Pictures  of  terrain  were  confusing 

1 

1 

1 

1 

Hap  graphics  were  poor 

l 

1 

1 

1 

Hap  was  impossible  to  read 

1 

1 

1 

2 

2 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(2) 

(2) 

(1) 

(3) 

(1)  (3) 

(table  continues) 
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Response  Cluster 


What  Student  Disliked  Host 


_ Event/Student  Characteristic 

Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Humber 

of  Responses  No  Yes  High  Low  High  Low 


Equipment  Problems 

Bugs  were  not  all  worked  out 

Computer  went  down 

(Number  of  Students 
Responding  to  Cluster) 

Miscellaneous 

No  substitute  for  the  field 

Not  applicable 

Nothing 

(Number  of  Students 
Responding  to  Clustei ) 

No  response 


1 

2 


1 


2 


1 


2 


1 

2 


(3)  (2)  (1)  (1)  (2)  (3)  (0) 


1  1 
1  1 
1 


1  1 
1  1 
1  1 


(3)  (2)  (1)  (0)  (3)  (3)  (0) 


(Number  of  Students) 


(2) 


(0)  (2)  (2)  (0)  (0)  (2) 


Table  6-3 


Number  of  Responses  to  ARTACT  Quest ionnai re  Item  Asking  What  Improvements 
Should  be  Made  in  ARTACT  (Presented  by  Response  Cluster  and  Event/Student 
Characteristic) 


Response  Cluster 


Improvements 


_ Event/Student  Characteristic 

Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number 

of  Responses  No  Yes  High  Low  High  Low 


Programs  Characteristics 

Expand  the  vocabulary 

Do  not  show  the  movement  techniques 

Work  more  on  offensive  and  defensive 
tactics 


1  1 

l 

1  1 


1 


1 


Make  It  more  challenging  1 

Increase  the  amount  of  detail  In  the 
Instruction  1 

Allow  deviations  from  doctrine  1 

Make  the  programs  more  Interactive  1 

Revise  the  programs  so  that  they 

do  not  assume  prior  knowledge  1 

Increase  the  flexibility  of  the  programs 
In  terms  of  accepting  answers  2 

(Number  of  Students 

Responding  to  Cluster)  (6) 


1 


1 


1 


1 


1  1  1 


1  I  1 


1111  11 
(3)  (3)  (5)  (1)  (2)  (4) 


(table  continues) 
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Response  Cluster 


Inprovements 


_ Event/Student  Characteristic 

Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number  ~ 

of  Responses  No  Yes  High  Low  High  Low 


Software/Hardware 


Use  more  reliable  equipment 

4 

2 

2 

1 

3 

1 

3 

Get  a  new  program 

1 

1 

1 

1 

Software  needs  revision 

1 

1 

1 

1 

Make  programs  more  user  friendly 
so  I/O  Is  not  needed 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(6) 

(4) 

(2) 

(1) 

(5) 

(2) 

(4) 

Map  Oisolav 

Inprove  graphics,  maps 

3 

1 

2 

2 

1 

3 

Use  paper  maps  with  legible  graphics 

1 

1 

1 

1 

Use  real  overlays 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(2) 

(2) 

(2) 

(2) 

(0) 

(4) 

(table  continues) 
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Response  Cluster 


Improvements 


_ Event/Student  Characteristic 

Experienced  Inprovement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Humber 

of  Responses  No  Yes  High  Low  High  Low 


Feedback 


Provide  more  detail  (e.g.,  more  detailed 

AAR)  when  pointing  out  mistakes 

1 

1 

I 

1 

Provide  prompts  that  will  not  give  away 
answers 

1 

1 

1 

1 

Have  somebody  familiar  with  school  doctrine 
present  to  discuss  answers 

1 

1 

1 

1 

Add  more  human  evaluation  that  points  out 
better  decisions 

1 

1 

1 

1 

Pronpts  should  be  timely  rather  than  at  end 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(3) 

(1) 

(4) 

(0) 

(2)  (2) 

Scenarios  and  Missions 

Focus  training  on  one  continuous  mission 
with  FRAGOs 

1 

1 

1 

1 

Need  more  combat  scenarios 

2 

2 

2 

2 

Need  realistic  situations,  Including 
casualties 

1 

1 

1 

1 

New  situations  should  be  presented  after 
feedback  Is  given 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(1) 

(3) 

(2) 

(2) 

(2)  (2) 

(table  continues) 
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% 


Event/Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledqe 

Improvements 

Number 

of  Responses 

No  Yes 

High  Low 

High  Low 

Uses  for  ARTACT 


Use  it  to  teach  offensive  and 

defensive  missions  1 

Use  it  to  teach  all  the  basics  (hand  and 
arm  signals,  movement  techniques)  1 

Should  be  able  to  practice  all  facets  of 
artillery  use  until  skills  are  mastered  1 

Should  Increase  Its  availability  1 

Use  to  supplement  tactics  Instruction 

prior  to  10-day  war  1 

(Number  of  Students  Responding 

to  Cluster)  (3) 

Consistency  with  Classroom.  Ooctrlne 

Hake  the  programs  more  In  line  with 

current  curriculum  1 

Make  sure  the  programs  are  doctrinal ly 
correct  1 

Do  not  require  gun  tube  to  be  to  the  rear 
when  the  tank  is  the  lead  element  In  a 
company  column  formation  1 

Make  the  programs  more  In  line  with 
classroom  Instruction  1 

(Number  of  Students 

Responding  to  Cluster)  (4) 


1  1  1 

1  1  1 

1  1  1 
1  1  1 

1  1  1 


(2)  (1)  (2)  (1)  (0)  (3) 

1  1  1 
1  1  1 

1  1  1 
1  1  1 

(3)  (1)  (2)  (2)  (2)  (2) 


(table  continues) 
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Event/Student  Character is t Ic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
In  Tactical 
Knowledge 

Improvements 

Number 
of  Responses 

No 

Yes 

High 

Low 

High 

Low 

Scheduling 

Schedule  the  programs  so  that  the  topics 
are  presented  after  they  are  covered  In  the 
classroom  1 

1 

1 

1 

Should  be  set  up  during  class  hours 

2 

1 

1 

2 

2 

(Number  of  Students 

Responding  to  Cluster) 

(3) 

(1) 

(2) 

(1) 

(2) 

(2) 

(1) 

Miscellaneous 

Use  the  ARTACT  programs  with  terrain 
boards 

1 

1 

1 

1 

Drop  ft 

1 

1 

1 

1 

(Number  of  Students  Responding 
to  Cluster) 

(2) 

(2) 

(0) 

(1) 

(1) 

(I) 

(1) 

No  Responses 

(Number  of  Students) 

(3) 

(2) 

(1) 

(1) 

(2) 

(3) 

(0) 
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Table  G-4 


Number  of  Responses  to  ART ACT  Questionnaire  Item  Asking  Students  to 
Write  Comments  and  Suggestions  (Presented  by  Response  Cluster  and 
Event/Student  Characteristic) 


Event/Student  Characteristic 


Response  Cluster 


foment  or  Suggestion 


Experienced  Improvement 

Equipment  In  Tactical 

Total  Problems  Class  Rank  Knowledge 

Number  '  “  . 

of  Responses  No  Yes  High  Low  High  Low 


ARTACT  Is  rigid  and  primitive  11  1 

Get  rid  of  the  program  and  the  civilians 

who  wrote  It  111 


ARTACT  should  not  be  part  of  AOBC  curriculum  11  1 

Spend  more  time  In  the  field  on  real  tanks  11  1 

Live  tactics  In  AOBC  more  interesting  and 

better  than  ARTACT  11  1 


Programed  text  would  be  better  and  cheaper  11  1 

ARTACT  Is  a  technology  looking  for  a  reason 

to  exist  11  1 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


(Number  of  Students 

Responding  to  Cluster)  (5)  (4)  (1)  (1)  (4)  (1)  (4) 

(table  continues) 
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Event/Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledqe 

Number 

Comment  or  Suggestion  of  Responses 

No 

Yes 

High 

Low 

High  Low 

Suqqestlons 

Should  teach  how  to  analyze  mission 
and  initiate  action 

1 

1 

1 

1 

Use  for  N60  students  while  AOB  students 
are  on  COFT 

1 

1 

1 

1 

Kill  the  user  if  he  makes  a  drastic  mistake 

1 

1 

1 

1 

Schedule  ARTACT  on  week  nights 

1 

1 

1 

1 

Use  ARTACT  to  teach  skills  and  terrain 
boards  to  restore  skills 

1 

1 

1 

1 

Have  Tactics  Branch  (MTT)  representative 
review  ARTACT  programs 

1 

1 

1 

1 

Have  tactics  instructors  scrutinize  program 

1 

1 

1 

1 

Should  be  allowed  to  go  back  after 
mistakes  have  been  identified 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(7) 

(3) 

(4) 

(2) 

(5) 

(4)  (3) 

(table  continues) 
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Event/Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledqe 

Comment  or  Suggestion 

Number 
of  Responses 

No  Yes 

High  Low 

High  Low 

Comnents  on  ARTACT  Staff  members 


ARTACT  staffmembers  are  flexible 
and  hardworking 

1 

1 

1 

1 

Staff  was  superb 

1 

1 

1 

1 

Fort  Knox  personnel  are  trying  to  build 
a  quality  product 

1 

1 

1 

1 

Enjoyed  the  instructors  who  were  helpful 
and  understanding 

1 

1 

1 

1 

ARTACT  personnel  were  cooperative  and 
flexible 

1 

1 

1 

1 

Working  with  professionals  is  a  pleasure 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(0) 

(4) 

(2) 

(2) 

(3) 

(1) 

Positive  Comnents 

Enjoyed  program,  learned  a  lot; 
enjoyed  thinking  about  actions 

1 

1 

1 

1 

All  AOB  students  should  participate  on 
ARTACT 

1 

1 

1 

1 

ARTACT  allowed  me  to  apply  the  curriculum 
in  a  practical  manner 

1 

1 

1 

1 

Outstanding  learning  experience 

1 

1 

1 

1 

Less  boring  and  more  efficient  than 
terrain  boards 

1 

1 

1 

1 

(Number  of  Students 

Responding  to  Cluster) 

(4) 

(Z) 

(2) 

(1) 

(3) 

(2) 

(2) 

(table  continues) 
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Event/Student  Characteristic 


Response  Cluster 

Total 

Experienced 

Equipment 

Problems 

Class  Rank 

Improvement 
in  Tactical 
Knowledqe 

Cooment  or  Suggestion 

Number 
of  Responses 

No  Yes 

High  Low 

High  Low 

Wo  Conments 


(Number  of  Students) 


(ID 


(9)  (2)  (7)  (4)  (6)  (5) 


